Artificial intelligence (AI) gained unprecedented attention recently with the rise of Generative AI brought about by the success of ChatGPT. Generative AI is a class of AI that has the ability to produce new content based on what it learns. Traditional AI, on the other hand, leans towards pattern recognition and decision-making for specific applications. My academic background, decades ago, centers on the latter.
Contrary to what many people believed, AI, is not your new kid on the block. It has been around for a long time before many of us are even born. In fact, not too long ago, before Generative AI became the buzz, there was an earlier euphoric wave that rides on developments in Big Data and Deep Learning. I have written about this in the past (you can read my earlier post here).
If you are confused by all this jargon mentioned, you are not alone. To make sense of them, think of it this way. ChatGPT is a specific application developed with Generative AI models. These models are trained using Machine Learning (ML) techniques of which Deep Learning is one. And, of course, to be able to train these models, you will need to be able to store and process very large sets of data, and that is where Big Data comes in. The confluence of technology advances in storage, data processing, learning algorithms, and the exponential growth of a wide range of data made possible what we see today.
While developments are still in progress, we already witnessed tremendous achievements. Today, AI can chat and respond almost like a human. It can code simple scripts in various languages such as Python. It can help to write articles, put up presentation slides, do poetries, produce art pieces, create videos, compose songs, convert speech to text or vice versa, beat the world champion at chess, solve complex problems, and more.
What about its potential applications in investing?
As for investors like us, we are more interested in knowing how AI will shape the investing landscape. What can it do for us and does that mean that humans will have less role to play in the future?
Rule-based expert system
The simplest and oldest form of AI probably exists as expert rule-based systems. In these systems, human expert knowledge in a specific domain is hardcoded into the computer as rules. You can picture yourself as the expert and someone asking you a long list of questions on what you will do if this and that happens under different scenarios. After that, he codes it into the computer. And if he can completely capture every rule you use precisely, then theoretically the computer would be able to do what you do.
But such a system can get awfully complex and is also rigid and difficult to maintain, not to mention that it is a big challenge to extract precise rules out of a human expert in the first place. With a highly complex function such as trading or investing where the experts are dealing with financial markets, uncertainties, biases, and emotions, it is an uphill task. Besides precise rules, they are often also guided by other non-quantifiable things or things they did not even realize they will consider until they are in that situation. You just have to check around and see for yourself if any two discretionary investors or traders, even if they share similar trading profiles and approaches, ever have the same performance consistently. I know of none. Even elaborate setups where experienced discretionary traders try to mentor or train successors in their approach usually end up with little success. No two persons are the same.
Machine Learning (ML) System
In terms of potential, the investment community is looking at AI systems that use Machine Learning (ML) algorithms. With ML, you do not need to prescribe any rules. The computer is left to define its own “rules”. It takes in the data as inputs and then iteratively develops “rules” that use these data towards solving the problem you want. One of the most widely used models is neural networks. Again, there are many variants of neural networks, but the most basic one is what we called a Multi-Layer Perceptron (MLP). And that is also one of the models I used in the past.
Multi-Layer Perceptrons
Neural networks model our brains as a set of interconnected neurons through at least 3 layers. The first layer is the input layer. That is where it receives external inputs just like how our brains receive information through our senses – what we see, hear, taste, smell, touch, etc. Then this information is passed through the next layer which is called the hidden layer. That is where it is processed. The rules are formed here through a mathematical relationship derived from these connecting neurons. It is powerful and flexible enough to model complex non-linear relationships without needing you to predefine how these relations work. But the tradeoff is it becomes a black box. All you have is a massive set of weights between the interconnecting neurons which is not very interpretable, unlike regression models where you can easily see and explain the relationships. Then in the final step, it passes to the output layer which does the final processing and gives us the outcome.
If you are trying to solve a very complex problem with huge amounts of different inputs and data, then more hidden layers may be used to improve the results. And when more than one hidden layer is employed, that is when we venture into the space of Deep Learning.
An example of the use of MLP for investing
As an example of how this might work, let’s say we want to build a model to predict the direction of where a stock index will head over the next 5 days. We feed the MLP with historical input data. For our purpose, the traditional inputs might be technical, fundamental, and economic historical data. These inputs are then fed into the hidden layers to produce an output. The output can be something simple like a value between 0 and 1 where a number close to 1 indicates a stronger likelihood of the index going up and a number closer to zero means chances lean towards going down. Then we match these outputs against the real outcome which is what we desired. The difference between the model’s output and the desired outcome is then used to back-adjust the weights between the neurons to bring it closer to getting the right outcome in the next round.
The model will not get it right in its first attempt. This process is done iteratively over thousands of rounds to train the model’s accuracy at predicting up to a stable and acceptable level. It is just like how we learned to spell a list of words when we were in school. We won’t get it right in the first round, so we just keep repeating until we commit the words reliably to memory. What I showed is probably one of the most direct ways how one can use AI for investing. But I am sure there are other more novel approaches.
Will Artificial Intelligence Bring About Significantly Better Investing Performance?
My view hasn’t changed from where it is years back. As far as using AI for investing and forecasting is concerned, if we strip out all the media hype and buzz, I think the reality will be still quite far from what people envisaged. And why do I say that? Didn’t AI already beat our top chess players consistently? So by that extension, shouldn’t it also be able to beat our top investment managers soon?
The financial market is not a simple game of chess
In 1996, Gary Kasparov lost to IBM’s DeepBlue in international chess. Then in 2016, Google's AlphaGo defeated Lee Sedol in Go. Both are well-known and celebrated milestones in the development of AI. But you cannot compare a game of chess to the financial markets. In chess, all you have is 2 players taking turns moving limited pieces on a chessboard one at a time. This is a well-structured problem where the rules don't change, and you know and can see all the historically relevant information i.e. how the pieces have been moved till now and where are their current positions. The number of possible moves you or your opponent can take is finite and we can calculate the best course of action.
Financial markets are in a totally different league. You have many millions of players. Information is everywhere and coming in every instant. Markets are constantly evolving as time passes, rewriting the rules on what works and what does not. You won’t have complete information because no one even knows what constitutes complete information. And not every piece will be relevant. Many are noises. Every player is independent. They will use what they have or want and they are capable of acting differently. And no, it is not a turn-by-turn game. No one is going to wait for you. By the time you act, things may have changed.
We are mining in a sea of data most of which will likely have no association with improving investment performance
There is an explosion of data today. So that opens up a world of opportunities for investors to access non-conventional or alternative data to see if they can get anything to help them make better forecasts for example. But unlike well-structured problems such as trying to identify the person speaking where there is a strong link between specific wave patterns and frequencies unique to that person, I know of nothing as definitive when it comes to investing. If you do factor investing, you will know that while some factors can explain historical investment performance very well, they fare much worse relatively when it comes to forecasting. And many data are correlated. So using more data will not necessarily yield better results. Because no matter how powerful your AI model is, it cannot extract any more information than what your data intrinsically holds. Having said that, of course, we cannot rule out someone finding some magic inputs among the sea of data today that might hold a stronger predictive power.
The risk of overfitting and the temptation to use garbage inputs is real
It is very easy to overfit an AI model by feeding it tons of data and using lots of different inputs. Just picture this. If I am trying to fit 2 points perfectly, all I need is a straight line on a 2-D space. If I am trying to fit 3 points that don't run in a straight line, I can do so with a plane in a 3-D space. Alternatively, I can still do it in a 2-D space by fitting a curve (aka non-linear solution) through it. And if I have more points than that, then I can either scale things into a hyper-dimensional space and/or combine it with non-linear fits. A neutral network is particularly suited for such tasks because it can map the inputs non-linearly into a higher-dimensional space to find a fit. That is why it is so powerful for pattern recognition or classification problems.
However, it is not something desirable to take this to the extreme, because it will produce overfitted models. Such models only work if what happens follows very closely to the past and leave little room for deviations. It will deliver spectacular performance in training, but it will fail miserably when put to work. You can liken this to training and conditioning someone to perform a task using a very rigid set of rules that must be followed to the dot with no room for exceptions. So, this person will do well if nothing deviates. But if he encounters even the slightest situation outside of what he was taught, he will flop. For investing purposes, what we need is not a perfect fit to the past, but a generalized fit. It is the same with any good education system where we want to produce students that not only has the knowledge but also the flexibility to handle and adapt to the real environment.
The other risk is feeding neural networks with all sorts of inputs including totally irrelevant garbage in the hope of finding something workable. For example, a person may use the positions of celestial bodies as inputs to predict stock market movements. As ridiculous as it might sound to you, there will be people that do that, and given the power of machine learning, it might find you something that seemed to work at least on training data.
Will The Advances In Generative AI Make A Difference?
I might be wrong as I don’t have much experience in this particular field of AI. But based on what I see, it can do what other traditional machine learning-based AI is already doing. For instance, it can be trained to do forecasting and portfolio optimization. But what sets Generative AI apart is its ability to produce new content from what it learned. So, it can potentially be used to create realistic market scenarios by generating its own synthetic data. You can then use these data to test how your investment models (these data can be applied to other models, it is not limited to AI models) would perform under different market conditions thereby allowing a deeper analysis and understanding.
But do all these translate to better accuracies at making forecasts, optimization, or risk management, and hence better investment performance? This is something that remains to be seen. Because it faces the same challenges I mentioned. At this juncture, we have not yet seen any groundbreaking academic discoveries for the use of AI on the investment front despite the vast resources now available. But, funds are latching on the wave, and launching new products around it. Whether this is a step sparked by genuine improvements or simply a business-motivated move, we will know the results in time.
Learn through our Free Financial Education Series over a cup of coffee!
We are running a series of short lessons and experience-sharing for those who are free to drop by the town area (around Raffles Place). Topics include various investment asset classes, options & futures, alternative investments, and what investors look for. The sessions are free and you can relax with a cup of coffee in a group setting. Simply click on the button below and register your interest. Choose the topics you are interested in and we will make the arrangements.
Disclaimer & Disclosure
We are not financial advisers or fund managers. The information published on this Site is provided for informational purposes only. It is not intended to be, nor shall it be construed as, financial advice, an offer, or a solicitation of an offer, to buy or sell an interest in any investment product. Nothing on this site constitutes accounting, regulatory, tax, or other advice.
Any performance shown on this Site is model performance and is not necessarily indicative nor a guarantee of future performance. You should make your own assessment of the relevance, accuracy, and adequacy of the information contained on this Site and consult your independent advisers where necessary.
AllQuant is carrying out introducing activities for iFAST Global Markets (Singapore) as an independent entity and is NOT an agent, servant, employee, representative, or in partnership with iFAST Global Markets (Singapore). AllQuant will be receiving remuneration or introducing fees from iFAST Global Markets (Singapore).
Comments