Big Data in Business: How to Select Your First AI Project

Artificial intelligence (AI) is revolutionizing nearly every industry, creating incredible amounts of economic value in the process. It is one of the greatest tools that have ever been available to businesses, as it provides valuable insights that were previously unattainable by humans alone. At the same time, it is also disrupting almost every domain and industry. And as Andrew Ng puts it, “AI is the new electricity.”

These reasons are why so many organizations embark on a data-driven transformation, which is followed up by the selection and execution of their first AI project. However, a vast majority of initial enterprise AI projects fail, and this failure can largely be attributed to selecting the wrong AI projects to begin with.

Before taking on a complex and highly rewarding AI project, your organization should prepare itself by educating the company, selecting and analyzing use cases, identifying desired outcomes, and assessing data characteristics to ensure that your first AI project has the best chance of success. If AI is indeed the new electricity, then every company needs new electricians.

Before reading on, make sure to check out the first two parts of this series:

The Importance of Leadership Understanding AI

When selecting your first AI project, it’s crucial for functional leaders to have a solid understanding of the ins and outs of AI. If they don’t, there is a high risk of the project falling through. But if they do, it will ensure that when you select your first AI project, everyone is prepared to carry it out with competence.

We’re not talking about total expertise here. They don’t need to be Elon Musk or Andrew Ng (although that would sure help).

In other words, they don’t need to learn how to code in Python, but they should most definitely know that Python is an important programming language for AI. It’s also important that they understand major AI concepts and technologies like machine learning, deep learning, and natural language processing (NLP), as well as how algorithms can leverage data and turn them into valuable insights.

A lack of domain knowledge and AI literacy can lead to leadership viewing the AI project as just another software project. This can dramatically affect the outcome. AI is a paradigm shift, not just in its technical aspects, but also cultural ones. AI projects are always multidisciplinary.

You could always go over some of my deep dive articles to help prepare leadership. Pieces like “What is Deep Learning?” and “What is Natural Language Processing?” are a strong introduction for those looking to get involved.

It’s also a great idea to have at least one “Introduction to AI” workshop to demystify the technology and prepare the company.

You can also refer to the many great resources available:

  • AI for Everyone: A Coursera class taught by Andrew Ng himself. It is aimed at helping non-technical colleagues become better at using AI.

  • Elements of AI: Offered by the University of Helsinki, this course is aimed at demystifying AI and teaching you what AI is, what is possible (and not possible) with AI, and how it affects our lives.

  • Introduction to Artificial Intelligence (IBM): Designed for anyone who wants to learn about AI with hands-on lab experience, IBM’s course requires no technical or programming background.

For those of you in the San Francisco Bay Area, I instruct an AI and business workshop at Stanford University. Part of the Stanford Continuing Studiesprogram, it is called “Artificial Intelligence and Machine Learning: Make Your Business More Effective and Profitable.” Reach out to learn more and inquire about next dates!

Selecting and Analyzing Use Cases

Before deciding on your first AI project, it’s important to select and analyze potential use cases. By identifying beneficial use cases, articulating desired business outcomes, and having a deep understanding of how AI is being used throughout your domain, you will have a better understanding of your AI project’s relationship to the larger industry.

Carrying out a strong case study ensures business leaders and decision-makers can understand a project’s end-to-end process while learning from the mistakes of others. And it shouldn’t just stop at one use case. You should study as many relevant applications as possible before embarking on your own, helping raise the chance of success.

When analyzing and selecting these use cases, you must stay determined and unwavering. It is often hard to determine how successful an AI use case is since the vast majority of them fail, making it even more crucial to be as prepared as possible. Some companies really struggle to define and prioritize AI use cases, and Gartner has estimated that 85% of AI projects will fail through 2022. By understanding use cases, you increase your company’s chances of being in that 15%.

The biggest risk is that you underestimate this critical step of the process!

To first identify AI use cases, you’ll want to identify areas of improvement and opportunity while targeting a business-critical problem. For example, this could be along your customer journey or involve your business process maps.

AI applications are often used to improve areas like:

  • Bottlenecks: The lack of cognitive insights can be caused by a bottleneck in the flow of information; knowledge exists in the organization, but it is not optimally distributed.

  • Scaling: In other cases, knowledge exists, but the process for using it takes too long or is expensive to scale.

  • Inadequate Firepower: Finally, a company may collect more data than its existing human or computer firepower can adequately analyze and apply.

After identifying potential use cases and opportunities, it’s important to prioritize them based on their value and ease of implementation. You should assess some specific dimensions like required data, complexity of required algorithms, required adoption of processes and systems, difficulty of scaling AI, and regulatory and ethics concerns. This process can be greatly improved by forging a business-to-IT alignment and bringing on experts from fields like data and machine learning.

The next step is to select the use case you’ll move forward with. While this may seem as simple as analyzing the results and picking the most attractive option, it is actually a more complex process that deserves more attention. It should be an iterative process where there is a rough assessment and prioritization, followed by reassessments, reviews, and a check for any potential red flags.

This “iterative development” process can be found throughout the lifecycle of an AI project, from selecting the use case to implementing the technology.

Source: IBM/DHL

In this phase, you’ll likely learn that some, or all, of the high-value use cases are extremely complex. It’s important that an organization’s first AI project is not too huge in terms of investment and resources. It’s always better to start with a small project that can deliver a quick win rather than a large one that risks draining the company’s time and resources. To overcome this risk, you should try to decompose the use cases into smaller approaches that have a higher chance of success.

Identifying Desired Outcomes

After selecting the use case for your first AI project, it’s time to identify the desired outcomes and which data analysis techniques are required.

The big data revolution has ushered in various different types of data analysis, so it’s important to understand each one to know which fits best with your selected use case.

The main types of data analysis are:

  • Descriptive Analysis: This is the most basic form of analysis, which is why a majority of organizations use it. You can think of descriptive analysis as answering the question “What has happened?” It analyzes historical data to represent what has happened in the past. It uses two key methods to analyze historical data — data aggregation and data mining — which help uncover trends and patterns. An example of descriptive analysis is traffic and engagement reports, where your organization tracks engagement from social media or web traffic.

  • Predictive Analysis: A more advanced method of data analysis, predictive analysis uses probabilities to assess what could happen in the future. It uses data mining along with statistical modeling and machine learning techniques to identify the likelihood of future outcomes based on historical data. Predictive analysis is often used to analyze and predict customers’ buying behavior.

  • Prescriptive Analysis: Relying heavily on mathematics, computer science, and various statistical methods, prescriptive analysis shows companies the best option to take. It is better equipped for actionable insights rather than data monitoring. Prescriptive analysis gathers data from a range of sources before applying them to the decision-making process, and algorithms create and re-create decision patterns that affect the business in different ways. A great example of prescriptive analysis is algorithmic recommendations, which is a result of business algorithms gathering data based on engagement history.

Besides identifying the best type of data analysis for the selected AI use case, you should also identify potential automated actions and processes. There are a few key aspects that make certain tasks better-suited for automation.

AI is often said to be best for repetitive or split-second tasks. While it is true that AI is highly beneficial for these tasks, it can also be used to automate many complex processes like extracting complex patterns, anomaly detection, and natural language generation (NLG).

For tasks that are described by logical rules that a machine can understand, AI is not necessary. However, applications that don’t contain these logical or exact rules benefit by intelligent automation. For example, a machine could be told to forward any email that contains “job application” to HR, but what if the email doesn’t include these words but still should be forwarded? AI could identify patterns that help it determine whether an email is a job application or not, even without specific target words.

The other major factor for automated actions is whether or not there are enough underlying data to teach the machine, since models require large data sets to operate properly.

Assess Data Characteristics

After selecting a use case, determining the right type of data analysis, and identifying potential automated actions and processes, the next step is to assess data characteristics. Data are the fuel driving AI technologies, and without quality data, you risk a failed AI pilot project.

I’ll keep this section somewhat short since the first part of this series, “Data Requirements for AI-Driven Businesses,” dives deeper into the topic of data.

You should ask some important questions regarding data and your first AI project:

  • What data exists internally?

  • Do we need external data?

  • Is it high-quality data?

  • How are the data organized?

  • What’s the readiness of our environment?

  • How can the data be applied to the selected use case?

  • What are the IT constraints on data accessibility?

The first thing you should do is look at the readiness of your company’s environment. Do you have a very little amount of data? Or too many data spread out across channels? If you don’t have sufficient data within the company, it’s time to look for external partners.

And after the selection of data is complete, they often need “cleaned,” referring to the intense process of formatting them to be consistent. Even though cleaning comes after the selection process, it’s important to stress that there’s some overlap, with cleaning being a component of data selection.

All of these steps would be greatly benefited by bringing on data scientists and other experts within the field.

Selecting the Right Technology

With all of that out of the way, you made it to the fun part! It’s time to select the right technology for your first AI project.

An entirely new toolchain is necessary to unlock the potential of artificial intelligence and machine learning, and make it operational and usable for developers and enterprises.

This can get a bit confusing for those without a deep knowledge of the field, which is yet another reason why it’s important to establish a dedicated AI and technology development team for your first AI project. (But it’s important to remember, you will continue to become more knowledgeable in these areas with each subsequent project.)

If the company’s team is still undergoing changes or being built up, there are many outside options for the company. By bringing on an external team, you can prevent the AI project from falling behind due to a lack of experience. The company’s staff can also focus on other tasks and projects, helping the company generate value faster. With that said, it is important to always stay focused on constructing a technical team that will be part of the organization. This will help with future projects.

There are many different frameworks and technologies that can be applied to different use cases. For example, if you are applying speech recognition or text summarization, you’ll look at machine learning frameworks like TensorFlow, while applications like language translation and chatbot development require frameworks like PyTorch. Most AI projects make use of these open-source components, not in-house ones.

Once you have decided on the right technology for the selected use case, it’s time to test out your first AI project in the real-world business environment. AI is transforming every industry, creating $13 trillion of GDP growth by 2030. The rise of AI means the time is ticking on launching your first AI project.

By preparing company leadership, selecting and analyzing use cases, identifying desired outcomes, and selecting the right technology, you can secure your company’s spot in this AI-driven world, and prepare your company for current success and future success with subsequent projects.

Make sure to look out for the next installment of this series, “The Lifecycle of an AI Project.”

Follow MVYL on Twitter, LinkedIn, and Instagram for AI-related content.

Sign up for MVYL partner Giancarlo Mori’s Medium blog.

Giancarlo Mori