🔮 A recap of 2023’s most important tech advances - Part 2

Welcome to this edition of Loop!

This is part 2 of our 2023 recap, but if you’ve missed part 1 you can view it here.

‏‏‎ ‎ HIGHLIGHTS ‏‏‎ ‎

• Summary of the year, for each of the top tech companies

• The biggest trends we saw in 2023

• Looking ahead to 2024

Let's jump in!

Here’s a quick summary of what the top tech companies have been up this year.

• Solidified their partnership with OpenAI

• Released new GenAI tools for enterprise customers

• Focus has shifted to developing small language models, as the costs of running LLMs are too high

• Building tools to support the adoption of AI agents

• ChatGPT has led the company to increasingly focus on their commercial offerings

• Built more tools to help individuals and companies deploy GenAI solutions

• Released GPT-4 and DALL-E 3

• Graduated from a research company that not many people knew about, to a huge name recognised by hundreds of millions

• Responded to OpenAI & Microsoft’s GenAI announcements with Bard and Gemini

• Restructured their research labs, merged Google Brain and DeepMind together

• DeepMind has regularly released details about their latest research - which is often in robotics, healthcare, mathematics, and other sectors

• Google are a lot further ahead with their AI technology, than many give them credit for - especially when it comes to DeepMind

• Continued with their extensive list of GenAI research projects

• Embraced the open source AI community, as they hope to reduce OpenAI and Google’s grip on the GenAI industry

• Partnered with Microsoft to bring their LLM, Llama 2, to enterprise customers on Azure

• Lead research into vision systems, which are key for their VR headset ambitions and the “metaverse”

• Announced the Vision Pro and showcased their impressive technical feat

• Aim to redefine what modern computing and entertainment looks like, with the Vision Pro

• Focus on how AI solutions can improve the customer’s experience - don’t worry as much about trends, instead look at how the tech can be applied effectively

• Research breakthrough on how LLMs can be run on low-powered devices

• Responded to OpenAI and Microsoft’s challenge, created their own GenAI tools for enterprise customers

• Supporting the enterprise use of open source models, as an alternative to OpenAI’s offerings

• Trialed humanoid robots in their warehouses

• Invested billions in Anthropic, who are one of OpenAI’s other competitors
  

• Broadened their horizon and research, beyond just image generation

• Now working on developing models across many of the core modalities - text, image, video, audio, and 3D assets

• Growing outside of the English-speaking market, following their development of Japanese vision-language models

1. The release of ChatGPT has started a GenAI gold rush

Many companies decided they needed to get in on the action and an obvious first move was to make a ChatGPT clone. These were often simple wrappers over the OpenAI service, with some basic guidance given to the model on how it should behave.

However, the problem was that there was suddenly a sea of companies who were already doing the same thing. Typically, these companies - who were often startups - had little to no moat to protect themselves.

Some of these startups were swept away later in the year, as the big tech companies released better features to meet the needs of their enterprise customers.

OpenAI’s Dev Day was particularly devastating for the vulnerable GenAI startups, many of whom saw their business model wiped out during the 30 minute event.

But when we look at the industry more widely, it seems like many companies today are still in the “exploration” phase of their journey to implement GenAI tools. This is partly due to just how fast the industry is changing, which can lead to managers feeling uncertain about what path to take.

As the likes of Azure and AWS start to showcase examples of how GenAI tools can be implemented, more companies will have the confidence to properly adopt these tools and build their own.

2. Big tech companies throw everything into releasing new AI tools

We also saw the top tech companies quickly brush off their previous concerns about AI safety.

Instead, focus has shifted on immediately releasing their tools to the public as quickly as possible. This shift in thinking is huge and should be something to take note of.

In 2022, both Google and Meta decided against releasing their AI image generators as they believed they posed serious risks to society. But following the massive success of ChatGPT, they have come under pressure to change their priorities.

Google has since released Imagen to their cloud customers, which allows them to generate new images, and are building new tools for advertisers to change how their product images look.

Meta have taken similar steps, but have instead embraced the open source AI community - as they hope to reduce OpenAI and Google’s grip on the GenAI industry.

It does seem like we have shifted from an environment where we would carefully weigh up the societal impact of releasing a powerful new technology, to one where it should be released as fast as possible - no matter the cost.

It’s a risky strategy, especially considering how Western governments are no longer reluctant to step in and regulate AI. Given that around 75% of democracies will be going to the polls in 2024, and there’s a much greater risk of AI misinformation campaigns, these companies would be wise to stress their focus on responsibly developing new AI tools.

3. Apple gives the AR/VR industry a much needed boost

For almost a decade, we have heard how Augmented Reality (AR) and Virtual Reality (VR) are the next big thing. However, it still hasn’t materialised. Microsoft and Meta have led each of these sectors, but they have since scaled back their investments this year.

Apple’s announcement of the Vision Pro got a lot of attention for its huge price tag, but it’s clear that they have something big here. They have rebranded AR/VR as “spatial computing” and have managed to create a user experience that completely trumps what we have today.

While it’s not targeted at the masses, Apple will be keen to build a strong app ecosystem for the device and can bring the price down over time. They have proved to be patient with the Apple Watch, which has since grown to over 100m devices, and they will likely do the same with Vision Pro.

Given that Apple has recently been successful in running Large Language Models on low-powered devices, it’ll be fascinating to see what new experiences that developers can make possible.

4. Rise of AI influencers and hype in the media

Keeping up with technology nowadays is hard, but it has been made even more difficult by the rise of AI influencers on social media.

To be clear, I view an AI influencer as someone who creates a lot of noise when a new tool is released, but doesn’t add any real substance to the debate.

Every day on LinkedIn or X, we have become inundated with posts that proclaim something new as “ground breaking”. But if everything that’s new and shiny is classed as ground breaking, then nothing truly is.

This can make people feel like they’re struggling to keep up with the technology, since it seems like there’s always something new released every day. It also makes it more difficult for us to spot the valuable tools that can actually help with our work, which is counterproductive.

My hope is that the online debate will become a bit more grounded in reality and therefore make it easier for us to identify tools that are genuinely useful, rather than some of the excessive hype we see today.

5. AI regulation is finally coming into focus

How we regulate AI and how it is used has been an ongoing conversation for years, although there has been very little meaningful action from leaders around the world.

That changed this year, with President Biden announcing an Executive Order on AI safety and the UK hosting their own summit on how to tackle AI regulation.

Leading figures in the field have also called for more to be done, such as Yoshua Bengio, Geoffrey Hinton, and Stuart Russell.

While there has been a lot of focus on the existential risks, such as hostile states using AI tools to boost their development of nuclear weapons, not enough has been placed on the more immediate risks - including disinformation campaigns on a scale that’s never been seen before.

As we enter 2024, in a year when most of the world’s democracies will hold elections, the calls for regulation will only grow louder.

Last year was an exciting one for tech, but there’s a lot for us to keep an eye on in the coming year:

1. Vision Pro - Apple has made big claims on how this new device will change how we interact with technology, even going as far to create a new term for AR/VR - called “spatial computing”.

   
   But the most exciting aspect is how this ties in with the explosion of Generative AI tools. While ChatGPT’s multi-modal feature is already pretty useful, where you can upload an image and ask questions, it’s easy to imagine how the Vision Pro could take this to the next level.

   
   Unsure how to fix something in your house? Just take a picture with the Vision Pro and ask ChatGPT to show instructions right in front of you.
   
   This leads nicely into my next point…

2. Multi-modal - For years, we have seen research by Google and Meta on how they are further progressing their multi-modal models.
   
   These models are able to take in different types of data - such as text, images, videos, or audio files - and then answer questions about them. Some can even take in one source and generate new content, for example uploading an image of a train would lead to the model producing an audio clip with the train’s horn blaring.

   
   This opens up a huge number of opportunities that simply weren’t possible before. To date, high costs have limited the roll out of these models. But given how fast OpenAI have been able to reduce the cost of running their text models, it’s possible we will see a lot more multi-modal options going forward.

3. AI agents - In some ways, this technology is already here - but few companies are aware of what agents can do.
   
   AI agents are bots that can be assigned a task, which they will then try to solve for you. The bots can do this completely independently, or they can ask you for your advice on how to proceed.
   
   This is interesting because it opens up a new avenue for us. To date, companies will use tools like LangChain to create structured prompts that work together. That’s great, but there are times when you need a less rigid approach - this is where AI agents come in.
   
   Microsoft’s Research team have made huge progress in this field already with their release of Autogen. AI agents are an emerging part of the GenAI sector and something to keep a close eye on in 2024.

4. Robotics & Generative AI - The huge tech companies are already conducting research on how LLMs can be used to train robots, with some fantastic results.
   
   Meta, Microsoft, and Google have demonstrated what’s possible here, with robots that can take in text commands and pick up different objects - or projects that show how they can use Generative AI to more efficiently train these robots.
   
   But there’s a lot of potential here with multi-modal models, especially as they progress further and can take in more data sources than is possible today. While it’s a challenging sector to commercialise in the short term, we should hopefully start to see an acceleration in research within this area.

5. Small language models - The cost of running large language models in your subscription services can be really, really high - as Microsoft have found out with GitHub Copilot.
   
   It’s believed that every month they lose around $20 per user. But for those who often use the service, the losses can be as high as $80 per user.

   
   This has pushed Microsoft to develop smaller language models instead, as they aim to cut costs. It’s unlikely that they’re the only company facing this issue, so I only expect this trend towards smaller models to accelerate as more companies integrate them into their services.

This was a much deeper dive than what we’ve done before, so I hope you found it useful.

We’ve covered a lot in these recaps, such as:

• How ChatGPT has changed both the tech industry and the public’s perception of AI

• Timeline of the biggest technology advances

• A summary of the year, for each of the top tech companies

• The major themes in 2023

• And what we can look forward to in 2024

There’s no doubt that 2023 was a huge year for tech. While there were lots of interesting advances, the year was completely dominated by Generative AI - from large language models, to huge improvements with image and video generators, to audio tools that can replicate a person’s voice and make it sound like they’re singing “Viva La Vida”.

We’ve seen a huge amount of progress in such a short space of time - and 2024 is likely to be no different.

Have a good week!

Liam

Feedback

Share with Others

If you found something interesting, feel free to share this newsletter with your colleagues.

About the Author

Liam McCormick is a Senior Software Engineer and works within Kainos' Innovation team.

He identifies business value in emerging technologies, implements them, and then shares these insights with others.