I went to go install it this morning to check it out, but I had to decline when I read the privacy policy. I might check it out on my desktop where I have a lot more tools to ensure my anonymity, but I’m not installing it on my phone. There is not one scrap of data you generate that they aren’t going to hoover up, combine with data they get from anyone who will sell it to them, and then turn around and resell it.
I’m sure other apps are just as egregious, which is one reason I’ve been deliberately moving away from native apps to WPAs. Yes, everything you can possibly do on the internet is a travesty for privacy, but I’m not going to be on the leading edge of giving myself to be sold.
it’s actually pretty easy to run locally as well. obviously not as easy as just downloading an app, but it’s gotten relatively straight-forward and the peace of mind is nice
check out ollama, and find an ollama UI
That’s not the monster model, though. But yes, I run AI locally (barely on my 1660). What I can run locally is pretty decent in limited ways, but I want to see the o1 competitor.
that’s true - i was running 7b and it seemed pretty much instant, so was assuming i could do much larger - turns out only 14b on a 64gb mac
figured i’d do this in a no comment since it’s been a bit since my last, but i just downloaded and ran the 70b model on my mac and it’s slower but running fine: 15s to first word, and about half as fast generating words after that but it’s running
this matches with what i’ve experienced with other models too: very large models still run; just much much slower
i’m not sure of things when it gets up to 168b model etc, because i haven’t tried but it seems that it just can’t load the whole model at once and there’s just a lot more loading and unloading which makes it much slower
Not a word on Chinese models being censored in the article. What an odd omission.
It should also be pretty obvious that this is following the usual Chinese MO of using massive state subsidies to destroy the international competition with impossibly low dumping prices. We are seeing this in all sorts of sectors.
Okay… But isn’t it also possible that AI is massively overvalued and this is a more reasonable price point for the technology?
Overvalued - as in, less useful than it seems to be - probably, but the costs of running it are immense and they are certainly not that much lower in China (despite low energy prices due to nonexistent environmental standards), given the hardware embargoes they are under, forcing them to use less efficient hardware.
Interesting about Chinese energy market is that in recent decades they’ve been investing heavily in solar power. Once they’ve figured out grid energy storage, running LLMs shouldn’t be a problem anymore.
Another option is to skip most of the grid storage and just spam solar panels. Rely on batteries only to get you through the night, not to bridge power across seasons. Build enough panels that your country can meet its needs even on a cloudy day in winter. Then you have reasonable power costs in the winter and nearly free electricity the rest of the year.
You could see a lot of energy-intensive industries becoming seasonal. We have a crop growing season, a school season, and sports seasons. Why not an “AI model training” season?
That would be possible, but seasonal production has some serious drawbacks.
Let’s say you have a steel mill with several production lines, solar powered arc furnaces, and enough batteries to keep production running through the night. During the summer you can continue production 24/7, but in the winter you’ll have to shut down completely, because there’s not enough energy to keep even a single production line running. This means that there will be wild fluctuations in a variety of things:
- number of employees on site
- rate of steel produced
- demand for storage space for raw materials and steel products
- demand for logistics
- demand for maintenance
This means, that in order to deal with the fluctuations, you would need to have lots of spare capacity in pretty much everything: More machines, more people, more money. If you could keep the production steady throughout the year, you could do so with less. Also, what will the employees do during the winter? The skiing resorts can’t possibly employ all of them.
In the winter you’ll have plenty of time to fix anything that’s broken, but if there’s an unscheduled shutdown during the summer, you’re suddenly going to need lots of maintenance personnel and materials. Incidentally, those would be in short supply in the summer, because all the other factories would have the same problem. You would need to have lots of spare capacity in maintenance as well.
The AI industry should be fine, since you could train models when energy is cheap. Oh, but what if the summer isn’t long enough for you to update all your models? Simply just buy more computers so you have more spare capa… Oh, it’s the steel mill problem all over again. Oh, but what about the people who use the models during the winter? Maybe you could charge your customers double the price during the winter so that the traffic would be reduced to a reasonable level. Fortunately though, wind power and other renewables could help with the winters, but having more grid energy storage would make things run smoother.
Honestly, this all seems like small potatoes. We’re trying to save our species from extinction here. We’re trying to maintain the standard of living that came with the Industrial Revolution without burning out planet to a cinder.
If doing so means our steel industry runs 10% less efficiently, I really don’t give a damn.
At to end of the day it comes down to this:
Is it cheaper to store steel stock in a warehouse or terrawatt-hours of electricity in a battery farm?
Is it cheaper to perform maintainance on 2 or 3x the number of smelters or is it cheaper to maintain millions of battery or pumped hydro facilities?
I’m sure production companies would love it if governments or electrical companies bore the costs of evening out fluctuations in production, just like I’m sure farmers would love it if money got teleported into their bank account for free and they never had to worry about growing seasons. But I’m not sure that’s the best situation for society as a whole.
EDIT: I guess there’s a third factor which is transmission. We could build transmission cables between the northern and southern hemispheres. So, is it cheaper to build and maintain enormous HVDC (or even superconducting) cables than it is to do either of the two things above? And how do governments feel about being made so dependent on each other?
We can do a combination of all three of course, picking and choosing the optimal strategy for each situation, but like I said above I tend to think that one of those strategies will be disproportionately favorable over the others.
i believe one of the big advancements with deepseek r1 is their method of adding the reasoning component is novel and very very efficient. i haven’t checked it out, but it could legitimately just be more efficient to run
Overvalued - as in, less useful than it seems to be
Uh, no… “Value”, as in quality/performance for the price. You can literally overpay for anything in this world, just look at the luxury market.
I’m not claiming to know enough about AI or LLMs, but I don’t think the first to market or the most prominent always set the price. So I think we’ll have to see what the accepted price actually turns out to be…
Energy costs money in China too, they still have coal plants and crazy energy cutback mandates every once in a while.
The truth of the matter is that you need the user interactions from the free model to train and that value cannot be understated and if you are playing catchup, then it’s a must. No one would use the Chinese model if it was a shit service.
It should also be pretty obvious that this is following the usual Chinese MO of using massive state subsidies to destroy the international competition with impossibly low dumping prices. We are seeing this in all sorts of sectors.
In this case, DeepSeek is announcing the training time for their LLMl, which wall street is extrapolating costs from. No state aid involved.
The article mentions the cost of tokens to end users, not training time.
Ah ok, I didn’t catch that. Other articles were discussing v3’s training using only 2.8M GPU hours.
https://www.ft.com/content/c82933fe-be28-463b-8336-d71a2ff5bbbf
Until this has been independently verified, I have my doubts. This wouldn’t be the first time for China to vastly exaggerate its technological capabilities.
tech has been subsidizing ai costs by magnitudes for years trying to make fetch happen, slop is slop. it’s overvalued like crazy and the first hint of market competition has drained trillions from the stocks because it’s an overvalued bubble. if china can do that by releasing competition then ok. maybe we should all be putting these trillions in things actually useful to humans.
If anything, this is just the start of an arms race. Do you really expect the Western competition to just stop what they are doing, because a single Chinese model performs well in a handful of synthetic tests that it was probably optimized to score well in?
I’m not a fan of AI slop either, on the contrary, but let’s be realistic here.
an arms race for what? more efficient slop? most of their value comes from the expected exclusivity - that say openai is the only one who can run something like o1. deepseek has made that collapse. i doubt they will stop doing stuff, but i dont think you understand the nature of the situation here.
also lol, “performs well in synthetic tests it was optimized to score well in” yes that literally describes every llm. Make no mistake: none of this has a real use case. not deepseek’s model, not openai’s, not apples, etc. this is all nonsense, literally. the stock market lost 2 trillion dollars overnight because something that doesnt have a use case was one upped by something else that also doesnt have a use case. it’s very funny.
Just read it also works on Huawei AI chips so you don’t even need anything Nvidia, might be they are going to take more hits.
The ‘cheap’ AI from China comes with a hefty price that’s not worth it.
Following @DdCno1@beehaw.org’s comment in this thread, the article appears to be the same Chinese propaganda narrative we haven been observing in the past. And it is misleading.
I have been posting this recently, and it perfectly fits here again. The base for China’s AI development is the so-called “AI Capacity Building and Inclusiveness Plan” which is aimed particularly at the Global South:
[Chinese] Government rhetoric draws a direct line between AI exports and existing initiatives to expand China’s influence overseas, such as Xi Jinping’s signature Belt and Road Initiative (BRI) and Global Development Initiative (GDI). In this case, the more influence China has over AI overseas, the more it can dictate the technology’s development in other countries […]
[According to the Chinese government] AI must not be used to interfere in another country’s internal affairs — language that the PRC has invoked for as long as it has existed, both to bring nations of the global south on board in China’s ongoing efforts to seize Taiwan and to deflect international criticism of its human rights record […]
[For example] China’s decision to co-launch its AI Capacity Building plan with Zambia also had a symbolic element. PRC state media reported that the African nation was the recipient of thousands of Chinese workers and hundreds of millions of RMB in loans in the 1960s, making it the beneficiary of one of China’s earliest overseas infrastructure projects — another thread connecting the latest in AI cooperation with China’s long-held ambitions to lead the developing world, even as it becomes a superpower in its own right. In a 2018 meeting with the Zambian president, Xi said they must jointly “safeguard the common interests of developing countries.” […]
If it’s cratering US AI stocks, that’s it’s job. Destabilizing the economy of an enemy nation when it’s such a monoculture is SoP.
Good; I hope all of the AI stocks come tumbling down.
