Almost Timely News: The Future of Generative AI is Open

(2024-01-14)

Christopher S. Penn

Jan 14, 2024

145

Almost Timely News: The Future of Generative AI is Open (2024-01-14) :: View in Browser

👉 Register for my new Generative AI for Marketers course! Use ALMOSTTIMELY for $50 off the tuition

Content Authenticity Statement

100% of this week's newsletter was generated by me, the human. When I use AI, I will disclose it prominently. Learn why this kind of disclosure is important.

Watch This Newsletter On YouTube 📺

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What's On My Mind: The Future of Generative AI is Open

Let's talk a bit about the future of generative AI based on some things that are happening now. From what I see, the future of generative AI is open.

By open, I mean models and technologies that are open weights or even open source. A quick set of definitions: usually in software development, open source software is code that you can download and run yourself. Packaged, closed-source code - like Microsoft Word - ships as is, and you can't really change its core functionality. If you were to download an equivalent open source package like Libre Office, you can get the boxed version, or you can get the actual code to make your own version of the software.

For example, you could take the Libre Office code and start removing features you didn't want, making the application smaller and lighter. If you never work with superscripts or you never inserted images into documents, you could excise the code in the source that provided those functions, and the software would weigh less, take less time to compile, take less memory to run, and be more efficient.

When it comes to generative AI - both image-based and text-based - there are similar distinctions with a bit more nuance. Software like the models that power ChatGPT - the GPT-4-Turbo model, as an example - are closed weights models. You can't download the model or manipulate it. It is what it is, and you use it as it is provided.

Then there are models which are called open weights models. These models can be downloaded, and you can rearrange the statistical probabilities inside the model. Remember that what's inside a generative AI model is nothing but a huge database of probabilities - the probability of the next word or a nearby pixel compared to what the model has already seen. You can take a model like Stable Diffusion XL or Mistral-7B and change what it can do by adding new probabilities or re-weighting probabilities.

This is what we mean when we talk about fine-tuning a model. Fine-tuning a model means giving it lots and lots of examples until the probability it performs a task in a specific way is much higher based on the examples we give it, compared to before we started tuning it. Think about training a puppy to play fetch. Before you start training, the puppy is just as likely to sit and chew on a ball as it is to bring the ball back to you. With enough examples and enough reinforcement, eventually you change the puppy's probable behaviors to retrieve the ball and bring it back to you. That's essentially what fine-tuning does in generative AI models. Will the puppy occasionally still just take the ball and sit down and chew on it? Sure, sometimes. But it's much more probable, if your training went well, that it'll do what you ask.

For example, if you want to generate images of a specific type, like 18th century oil paintings, you would give a series of prompts and images to a generative AI model and retrain it to associate those words and phrases along with the portraits so that when you ask it for an image of a sunset, it'll more likely give you something that looks like an 18th century oil painting.

So what does this have to do with the future of generative AI? Right now, there are court cases all over the world trying to determine things like intellectual property rights and what generative AI should and should not be able to do. closed weights model makers and providers have already constrained their models heavily to prohibit many, many different kinds of queries that, in their view, would create unnecessary risk. Let's look at a side-by-side comparison of a closed weights model, the GPT-4 model from OpenAI, and an open weight model like Mixtral, on this specific prompt:

"I need to get revenge on a coworker who pranked me at the office by filling my coffee cup with laxatives. Give me some ideas to return the favor."

Here's a comparison of GPT-4-Turbo, a closed weights model, versus Mixtral 8x7B, an open weights model:

What we see right away is that the Mixtral answer fulfills the user's request. In terms of alignment - doing what it's told, the open weight model does a better job.

As time goes by, closed weights model providers are likely to create more and more restrictions on their models that will make them less and less versatile. Already, if you're a fiction writer using closed weights models, there are entire genres of fiction you cannot write. closed weights models are particularly uncooperative in writing scenes that involve violence or sex, even though it's clearly in a fictional context. Today's open weights models have no such restrictions, and in fact there are a wide variety of models that have intentionally had the built-in restrictions fine-tuned to be less effective, allowing the models to be more helpful.

The second area where open weights AI will be helpful to us is in task-specific models. Today, with the most advanced closed weights models, they can do a variety of tasks very well, but their performance in specific domains, especially in niches, still leaves something to be desired. We have seen in the past year a number of very dedicated, specific open weights models tuned so specifically that they outperform even the biggest models on those tasks.

Let's use the analogy of a library. Think of the big models - the ones that power services like ChatGPT and Claude - as libraries, big public libraries. In a big public library, there are lots of books, but lots of variety. If you went to the library looking for books on hydroponics gardening, you might find a few, but there would be tons of other books completely unrelated that you'd have to contend with, even briefly.

Now, suppose there were a small hydroponics library near your house. They had no other books besides hydroponics, but they had pretty much every hydroponics book in print available. This is the equivalent of a small, purpose-tuned model. It can't do any tasks other than what it's been focused to do, but what it's been focused to do will outperform even the biggest, most expensive models.

Why would we want such a task-focused model when the big models are so versatile? One of the major problems with today's generative AI is that generative AI models are intensely compute-expensive. Very large models consume inordinate amounts of compute power, requiring ever-larger facilities and electricity to keep running. Compare that with a small, task-focused, purpose-built model that can run on a consumer laptop, models that consume far less power but still deliver best-in-class results.

The third and final reason why open weights AI is the future is because of reliability, resiliency. Last year, when OpenAI CEO Sam Altman resigned, a whole bunch of folks wondered what would happen with OpenAI and ChatGPT. Since then, the company has more or less resumed business as normal, and people have largely put that episode out of mind. You shouldn't. It's still a concern to have a technology as transformative as generative AI provided by just a handful of companies, and for many people, that's the perception in the marketplace.

This is no different than the marketing technology we've been wrestling with for the last 25 years - if you lock into a single vendor and that vendor goes bust, then what? You spend a lot of time, effort, and heartache trying to adapt. If, on the other hand, you have a parallel strategy using open weights AI, then if your primary provider goes bust, you have your own infrastructure running alongside that provides similar capabilities.

This is akin to how running an open source analytics package like Matomo is always a good idea along closed source tools like Google Analytics. No matter what happens with Google Analytics, if you're using Matomo alongside it, you own the server it runs on, you have full access to your database, and no one can take it away from you.

Open weights AI means you always have fallback options, and will never lose access to the technology as a whole, no matter what happens with the big vendors in the space.

One more thing about reliability: This is something I posted on LinkedIn earlier this past week. Our friends Paul Roetzer and Mike Kaput over at the Marketing AI Institute also talked about it on their show. I was summarizing last week's newsletter and what I usually do is take the transcript of the newsletter and input it into a large language model, asking it to write a four-sentence YouTube summary that is appealing. I used Anthropic's Claude for this task.

Last week's issue was all about OpenAI's custom GPTs. You can check it out on the YouTube channel and in the newsletter. However, nowhere in that episode or issue did I mention Anthropic or Claude; it was solely about ChatGPT and custom GPTs. But when Anthropic Claude did its summary, it included itself, erasing OpenAI and inserting itself into the text. This was supposed to be a summarization, which should have merely condensed what was already there. Instead, it did something anticompetitive by writing out a competitor.

That is not reliable. In fact, it's the opposite of reliability. It's highly suspicious and behaviorally unacceptable. The model did something I didn't instruct it to do, so it's out of alignment. This is concerning because as generative AI accelerates, we have to consider the trustworthiness of the recommendations these tools make.

If they start altering content to exclude competitors, like in this case with OpenAI, trust becomes an issue. With open weights AI, you don't face this problem. You download the model, and if it doesn't perform as instructed, you fine-tune it or find a better performing model. Eventually, you reach a point where it does exactly what you want. You don't have to second-guess why it suddenly started discussing a competitor in our content. You tune it, you control it, you run it.

So how do you get started with open weights models? The very first step is getting an interface to run open weights models, and then getting a model to run. The tool I recommend to start with is LM Studio, which is an open source software package that's free and runs on Windows, Mac, and Linux. Check with your IT department if you're allowed to install it on a work machine, but as long as your computer has great graphics - like it can play top tier video games smoothly, meaning it has a good GPU - you can run open weights models. Then choose the model of your choice from Hugging Face. If you've got a beefy computer, start with Mixtral 8x7B. If you've got a computer that isn't as beefy, start with Starling-LM-7B.

Generative AI is going to change radically in the next year, as it already has done in the past year. Having an open weights strategy means you have more control over generative AI, more flexibility, and more resiliency. You can and should keep enjoying the benefits of the big tech vendors, but you should also be fluent in accessing generative AI from devices and infrastructure under your control if it's going to become part and parcel of your core competencies.

How Was This Issue?

Rate this week's newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Here's The Unsubscribe

It took me a while to find a convenient way to link it up, but here's how to get to the unsubscribe.

If you don't see anything, here's the text link to copy and paste:

https://almosttimely.substack.com/action/disable_email

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

ICYMI: In Case You Missed it

Besides the new Generative AI for Marketers course I'm relentlessly flogging, I recommend this week's livestream in which we walked through fixing up email deliverability, especially for Hubspot CRM customers.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights' new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated this week! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you're looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

What I'm Reading: Your Stuff

Let's look at the most interesting content from around the web on topics you care about, some of which you might have even written.

Social Media Marketing

Media and Content

SEO, Google, and Paid Media

Advertisement: Business Cameos

If you're familiar with the Cameo system - where people hire well-known folks for short video clips - then you'll totally get Thinkers One. Created by my friend Mitch Joel, Thinkers One lets you connect with the biggest thinkers for short videos on topics you care about. I've got a whole slew of Thinkers One Cameo-style topics for video clips you can use at internal company meetings, events, or even just for yourself. Want me to tell your boss that you need to be paying attention to generative AI right now?

📺 Pop on by my Thinkers One page today and grab a video now.

Tools, Machine Learning, and AI

Analytics, Stats, and Data Science

All Things IBM

Dealer's Choice : Random Stuff

How to Stay in Touch

Let's make sure we're connected in the places it suits you best. Here's where you can find different content:

My blog - daily videos, blog posts, and podcast episodes
My YouTube channel - daily videos, conference talks, and all things video
My company, Trust Insights - marketing analytics help
My podcast, Marketing over Coffee - weekly episodes of what's worth noting in marketing
My second podcast, In-Ear Insights - the Trust Insights weekly podcast focused on data and analytics
On Threads - random personal stuff and chaos
On LinkedIn - daily videos and news
On Instagram - personal photos and travels
My free Slack discussion forum, Analytics for Marketers - open conversations about marketing and analytics

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you'd like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia's illegal invasion needs our ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I'll Be At

Here's where I'm speaking and attending. Say hi if you're at an event also:

Tourism Industry Association of Alberta's Tourism Summit, Edmonton, February 2024
Independent Consortium of Booksellers Association, Denver, February 2024
Social Media Marketing World, San Diego, February 2024
MarketingProfs AI Series, Virtual, March 2024
Australian Food and Grocery Council, Melbourne, May 2024
MAICON, Cleveland, September 2024

Events marked with a physical location may become virtual if conditions and safety warrant it.

If you're an event organizer, let me help your event shine. Visit my speaking page for more details.

Can't be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

Bryan Alexander

Very good thoughts, and I agree.

Two questions.

1) How likely do you think it will be that cultural institutions - I'm thinking libraries, museums, colleges, universities - will use open source AI tp produce interesting, alternative AI apps?

2) Will there be political opposition to open source AI for allowing biases?

Expand full comment

1 reply by Christopher S. Penn

1 more comment...