- Goldman Sachs pour water on investment payoff. (My piece here went a little nuts on Linkedin)
- WSJ say that investors are getting cold feet on AI plays. 
- Pharma CIO says Microsoft's AI tools in MS products are no better than a middle schooler and cancels all subscriptions.... 
- 28 Law Suits in the US against AI and copyright theft.
- Getty suing Stability AI for $1.8Tr.

I think the short term gains are over. Now who's shorting NVIDIA? 

There were two major winters approximately 1974–1980 and 1987–2000,[3] and several smaller episodes, including the following:
 
1966: failure of machine translation
1969: criticism of perceptrons (early, single-layer artificial neural networks)
1971–75: DARPA's frustration with the Speech Understanding Research program at Carnegie Mellon University
1973: large decrease in AI research in the United Kingdom in response to the Lighthill report
1973–74: DARPA's cutbacks to academic AI research in general
1987: collapse of the LISP machine market
1988: cancellation of new spending on AI by the Strategic Computing Initiative
1990s: many expert systems were abandoned
1990s: end of the Fifth Generation computer project's original goals
Enthusiasm and optimism about AI has generally increased since its low point in the early 1990s. Beginning about 2012, interest in artificial intelligence (and especially the sub-field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment, leading to the current (as of 2024) AI boom.

AI in 1972. btw not Ronnie Corbett

The post-1969 AI winter, following the publication of "Perceptrons"? It was ludicrous and says a lot about the domain - should be a warning to its practitioners. Minsky and Papert aimed to analyse the limitations of single-layer perceptrons (particularly the XOR problem) with mathematical scrutiny "in the name of science". In the very same work, they explicitly mention that a multi-layer version could overcome these limitations. But the community was completely out of tune with their scrupulous research style and used it blindly against the idea of neural networks altogether. The true problem they couldn't solve, though, was training...

This brings in a slightly less domain-specific paradox: the backpropagation algorithm was already described in the literature at the time (i.e. long before the seminal 1986 paper which popularized it and earned the authors many awards, incl. Turing / Nobel for computing):
Paul Werbos (1974): Ph.D. dissertation at Harvard University, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences"
Bryson and Ho (1969): "Applied Optimal Control: Optimization, Estimation, and Control"
Kelley (1960): technical report "Gradient Theory of Optimal Flight Paths"
(and probably many others thought about it, since everyone knows the Leibnitz rule!)

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

There were two major winters approximately 1974–1980 and 1987–2000,[3] and several smaller episodes, including the following:
 
1966: failure of machine translation
1969: criticism of perceptrons (early, single-layer artificial neural networks)
1971–75: DARPA's frustration with the Speech Understanding Research program at Carnegie Mellon University
1973: large decrease in AI research in the United Kingdom in response to the Lighthill report
1973–74: DARPA's cutbacks to academic AI research in general
1987: collapse of the LISP machine market
1988: cancellation of new spending on AI by the Strategic Computing Initiative
1990s: many expert systems were abandoned
1990s: end of the Fifth Generation computer project's original goals
Enthusiasm and optimism about AI has generally increased since its low point in the early 1990s. Beginning about 2012, interest in artificial intelligence (and especially the sub-field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment, leading to the current (as of 2024) AI boom.

Always fun when I talk about the next AI winter and I get laughed at (or even boo'd) at conferences. There was always going to be a time when Gartner's Hype Cycle and reality collided at full speed. 

Oh look the VIX is up again..... 

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

There is a large grain of truth in that, along with storage costs collapsing over the years. The lack of data for training was another, we actually cared about our data at the time, plus it wasn't that big to train any really ANN of size and get wonky results. 

Odd that no one talks about overfitting models now.... i miss those days. 

Now some of the world has gone mad over the Overfitted Stochastic Language Model™ 

Not even the Labour Government are sure about the money going into it now: https://www.bbc.co.uk/news/articles/cyx5x44vnyeo

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

And in 2024: slow progress with no limitations in computational power?
Chickens have come home to roost?

Is the game up?

just a question

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

And in 2024: slow progress with no limitations in computational power?
Chickens have come home to roost?

Is the game up?

just a question

I don't think the game will be up, just massively re-adjusted. The key will be to watching spot prices of GPU based instances on Azure etc. If there's a flood of capacity (ie no one is training anything) then NVIDIA just pull the best trick in the book of convincing everyone needed GPUs, but sales forecasts will decline and so the share price (by my pragmatic mind). Watching what all the lawsuits and legal action could do if the next part. Seeing how the EU AI laws pan out for large models might be an indicator too.

There hasn't been a decent use case and I can't really see one. Once you threaten to replace jobs then it gets messy and people rebel a bit. 

I got $3bn investment in my AI idea and all I've got left is this scarf which will keep me warm during the AI winter.......  

Multi level marketing 2.0?

Multi level marketing 2.0?

Isn't that giving MLMs a bad name?  

And I missed this yesterday as I'm not a FT subscriber and able to write it off on expenses....  
https://www.ft.com/content/24a12be1-a973-4efe-ab4f-b981aee0cd0b

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

And in 2024: slow progress with no limitations in computational power?
Chickens have come home to roost?

Is the game up?

just a question

I see progress in many dimensions. AI models have been rolled out in many real-life applications - it's no longer pretending that "we use AI" while secretly using Facebook Prophet (great simple explainable model, BTW!) or calculating string metrics. This shift enforced introducing legal and safety regulations and thus opening the black box to understand how NN models operate. For example, four years ago I used RNN model in a policy making project - people were skeptical but curious (open-minded folks). This year I revisited the problem with a new, also NN-based, approach (laughing at my old work), using a range of methods to interpret and explain the results. Needless to say, my previously AI-skeptical colleagues (with social science training) are now using reinforcement learning - and even tailoring the algorithm to suit their needs. It's a great progress that AI, until not long ago trapped in the world some math/CS weirdos, who sometimes can't figure out how to check the oil level without a manual or remember to stay hydrated, has been released into the real world, where intelligent people can shape it, use it and keep it under control.
Having said that, it's possible that there's been some over-investment and over-hiring in the AI sector. Still, as I mentioned before, AI is far more sustainable than eg dot-com, and I don't believe we will see any major burst. The market has many better reasons to crash now, anyway (not that I have the credentials or any detailed knowledge to predict such events). On the other hand, who knows... - we've been just discussing the absurd 1969 winter!

Yay! I'm rambling

IMHO (and I think most believe it), the true reason for the slow progress in AI research at the time was the limitations in computational power.

And in 2024: slow progress with no limitations in computational power?
Chickens have come home to roost?

Is the game up?

just a question

I see progress in many dimensions. AI models have been rolled out in many real-life applications - it's no longer pretending that "we use AI" while secretly using Facebook Prophet (great simple explainable model, BTW!) or calculating string metrics. This shift enforced introducing legal and safety regulations and thus opening the black box to understand how NN models operate. 

Having said that, it's possible that there's been some over-investment and over-hiring in the AI sector. Still, as I mentioned before, AI is far more sustainable than eg dot-com, and I don't believe we will see any major burst. 

Yay! I'm rambling

I just can't see companies wanting to break open a black box, they'd rather restrict access to countries where the law is not their favour (ie: Europe but not the UK). A general over hiring happend in 2020 and the talent flush has been happening fairly steady over 2023/24. There's not been an over hire in AI "talent" because basically there wasn't that much there to start off with. 

I purposely stuck with "machine learning" as not to get tied up with the AI influencer crowd. Though it's been nice to be able to point to citations and patent citations when someone decided to argue about what I knew

There's too much reliance on software as a whole to have a dotcom bust as in 1999/2000 but things go in cycles so it might be worth keeping an eye on. History is both a blessing and a curse.

In the immortal words of Tess Daly and Claudia Winkleman: Keep Rambling! lol

I see progress in many dimensions. AI models have been rolled out in many real-life applications - it's no longer pretending that "we use AI" while secretly using Facebook Prophet (great simple explainable model, BTW!) or calculating string metrics. This shift enforced introducing legal and safety regulations and thus opening the black box to understand how NN models operate. For example, four years ago I used RNN model in a policy making project - people were skeptical but curious (open-minded folks). This year I revisited the problem with a new, also NN-based, approach (laughing at my old work), using a range of methods to interpret and explain the results. Needless to say, my previously AI-skeptical colleagues (with social science training) are now using reinforcement learning - and even tailoring the algorithm to suit their needs. It's a great progress that AI, until not long ago trapped in the world some math/CS weirdos, who sometimes can't figure out how to check the oil level without a manual or remember to stay hydrated, has been released into the real world, where intelligent people can shape it, use it and keep it under control.
Having said that, it's possible that there's been some over-investment and over-hiring in the AI sector. Still, as I mentioned before, AI is far more sustainable than eg dot-com, and I don't believe we will see any major burst. The market has many better reasons to crash now, anyway (not that I have the credentials or any detailed knowledge to predict such events). On the other hand, who knows... - we've been just discussing the absurd 1969 winter!

Yay! I'm rambling