It seems like everybody and their mother has some new AI trend they want to make money off of these days. Jargon-filled phrases fly around like Natural Language Processing, Reinforcement Learning, Neural Nets, Big Data, and Machine Learning. It can be easy to get lost, especially because a lot of them are just variations on the same thing. I recently had a friend go through the application process for a company that specialized in 'cognitive AI'. Here's the breakdown of what Cognitive AI is, how to talk about it, and the right questions to ask of an employer when you're in the interview.

AI's major short-term private-sector value is best suited towards helping companies minimize the shortcomings of their current workload through the automatic ability to act on near-perfect information (i.e. observations from big data) without much human input. Most companies that focus on resource extraction or manufacturing suffer from over and under-production. If they have too much of a certain product, they can't make money off it. If they have too little of a product, that's lost revenue. Most companies also can't quickly shift production levels from high to low production - these are decisions that are made far in advance off of what production managers believe will be the long-term trends. It's the best strategy that these companies have - but could it be improved? Sounds like a perfect recipe for AI to fix.

The main thing that you'll want to talk about in any AI-related interview is data. Data collection, organization, housing, and processing are the four main attributes that you'll be interested it. It's really unfortunate that most companies are reactive to big data concerns than proactive - it's often an afterthought - but that's perfect for you to take advantage of. Most companies ignore these concerns because they don't understand them, or they've hired the machine learning engineers thinking that they'll be able to adapt (bad strategy). Data is essential to understanding the foundations of what your product is. Not having a thorough understanding of the data is like an architect not knowing what their building materials are. Ask your prospective employer what their data collection strategy is. How is it maintained? How is it processed for use in machine learning algorithms? These are big questions, and if they don't know the right answer - be prepared, and use it as an opportunity to show off.

Now onto why you clicked on this article. How does Deep Reinforcement Learning (otherwise known as Cognitive AI) actually work? What do you need to know to sound knowledgeable about it? Let's start with what it's used for. Using DRL, you can design automatic control functionality for robots, create theoretical worlds and fill them with potential factors to model future situations, allow machines to make decisions based on the world around them, and master game AI. The six main areas of DRL are:

1. Automatic Control and Imitation Learning
2. Reinforcement Learning
3. Deep Feedforward Networks
4. Reinforcement Learning in Large State Spaces
5. Policy Gradients and the Q Function
6. Reinforcement in Continuous Action Spaces

Each of these six subjects are vast enough to merit their own post (or several) on their own. You can read up on them anywhere online if you've got the time. We want to focus on how to talk about each of them. If you actually want to get into the technical aspects of Deep Reinforcement Learning, I recommend that you have prior experience with Deep Learning, Machine Learning, and a good understanding of Python programming with standard libraries like Numpy, Pandas, and Matplotlib.

If you're feeling a bit lost - don't panic! Everybody does when they get to this stuff. A good way to describe computer science is like a single tool that everybody and their brother managed to customize in their own unique way, and is desperate to get you to learn it. Nobody knows all this stuff at once, and even the most experienced machine learning engineers refer to the basics all the time. What's best is to generalize - and let's start generalizing with DRL. A good concept to grasp is the idea of Dynamic Systems. Dynamic systems are labeled as such because they're constantly changing. That's why they're useful - they can take in constantly changing data and do stuff with it! The second general concept is memory. Memory is important because that's all the information that your system can use to make decisions. Now - machines handle memory a bit different than you or I do. It's not like short-term memory vs. long-term for us. Instead, the metric that machines use as a data source is limited vs. unlimited memory. Each has differing reasons for its use. You want a dynamic system with limited memory if you don't want the information from a huge timeframe to interfere with the actions of the system (i.e. last summer had a record-breaking heat but you don't want your system to think that every summer is going to be like that!). Sometimes you'll want a dynamic system with unlimited memory to handle long stretches of time and produce outputs. It all depends on your use case - but remember! Computers have their limits too. If you try and store everything, you'll waste computational resources. The reason machine learning engineers get paid so much is because they have to determine which features are worth keeping in memory. Google Cloud gets expensive! Why did I introduce those two concepts over all the others? Dynamic systems really emphasize the point of deep reinforcement learning, which is to use the stimuli around the machine to get it to make a decision. You want the machine to take in everything around it, analyze all the data, and make a decision (i.e. the output).

The next big general concepts are predictive learning problems. There are two main predictive learning problems - regression and classification. These are terms that'll be familiar to any statistician or data scientist. How would you go about solving a regression problem? Take 'predicting the share price of a company about to go public.' You'll want to get a training dataset with known share prices, and try and design features that are thought to be relevant to your goal. This can be things like the company's current revenue (i.e. the higher the revenue, the higher the stock price). In order to connect the potential share price to the revenue, you'll use a linear model or regression line. These kinds of decisions are what DRL is ideal for. Using ML for classification is similar - but instead of predicting the future of an output, you're trying to predict a class (or a label). This kind of problem solving can be found in object recognition, where the machine learning model needs to distinguish an object from its surroundings and identify it. Mathematically, the way you do classification is by finding a hyperplane (i.e. a separating line) that separates the two classes of data from a training set as best as possible.

I'm only going to cover one more general concept before I go into some summarizations and ideal questions to ask with broad answers that you won't have to know everything to make it look like you've got this in the bag. The final concept is feature design - figuring out those defining characteristics of a given dataset that allow for optimal learning. The more you understand the process of generating the data that you have, the better you can design features yourself, or even better, teach the computer to design them itself. One of the fun end goals of machine learning is to develop adequate tools to understand our own universe - how to find patterns in otherwise arbitrary data and how to identify the best ones to use for significant observations.

One thing you'll probably want to discuss is how machine learning actually works. Above, I just listed the beginning tools to understand ML, but you probably want to know how they come together and actually solve a problem - right? Every machine learning problem eventually reduces to just finding the minimum of an associated cost function. If that sounds complex, it honestly is! A good way to think about this is how the brain has ridges, kind of like a canyon on a super small scale. That's where a funny thing happens - we've figured out that if we can shape the canyon right, the answer we want is always at the bottom! I can practically hear all the mathematicians groaning collectively at that explanation, but it's honestly a really good preliminary way to understand the 'magic' that goes on in machine learning systems. The real answer is more similar to the exercises you had to do in calculus where you were given a missing value in a series of linear equations (i.e. a matrix) and told to find the missing value. In this case, the missing value is the bottom of the canyon, and the canyon is the representation of the linear equations! If you're still confused about the abstraction, don't worry - it's a really weird concept for the human brain to grasp.

Want some good news? It's really easy to talk about machine learning in an interview if it's more theoretical than technical. A good way to stick to a script is to be honest about what's exciting about these kinds of technological breakthroughs - the potential of machine learning is really, really cool! That whole concept of using computers to find and make use of patterns in an otherwise chaotic universe should be fascinating for everyone. You're pretty much designing a virtual 'manager' with perfect information for a product if you're working with this in industrial production. What's better than that? I covered the idea of features above - you know what your features are? They're the manufacturing information - the number of people working, the acres of equipment in operation, the amount of energy used, the inputs and outputs. They're the situational awareness - the knowledge of the market, the knowledge of the surrounding area, the knowledge of what allows the inputs and outputs to come about. And both of those are combined to allow for perfect coordination. It's all one brain pretty much analyzing everything and making decisions. Cool right? If you want the key to talking about machine learning, just stick to what's positive about the whole process. The possibilities are endless.

Definitely inquire about the following three areas when you go into your interview. Depending on how confident you are in your knowledge of each area, alter and edit the questions to suit your needs:

1. Focus on data collection. Ask how statistically significant features are engineered by the company. Ask how those features are record and then processed to be of optimal use for the company's purposes. Not having data in the right format is the biggest problem. The single best question you can ask is how they label their data points. That will immediately clue your boss in that you know what the main issues are in machine learning. Unlabeled data is the bane of all machine learning engineers.
2. Ask about their use of cloud storage. Do they have adequate staff on hand to be able to deal with the complexities of managing a cloud data platform and pipeline to their services? Just having your intern attempt to launch a model on a GPU is not enough.
3. Ask whether their dynamic systems involved long-term memory or short-term memory, or a mixture of both. That will clue you in as to whether or not the company's thought intensively about their data collection practices and how they make use of them. A good follow up is what situations would you use both types of memory for.

Shoot me an email if you'd like more potential questions or things to know about deep reinforcement learning. It's a growing field and it's hard to find easy to grasp answers online. Check out the read next options I have below for more information. I didn't even cover implicit vs. explicit AI (the ability to understand what's going on in your system) or human-in-the-loop systems (responding to human input and making decisions). Those are your best next steps towards piecing together DRL.