It would help a lot if core Python libraries like urllib, NumPy, and others used SemVer. Removing a function is a breaking change. The root of this post is from urllib breaking something in a minor release. Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. Maybe after that the rest of the community could hope their code will work long term if they don’t increment major versions.
That just encourages people to keep using old, unmaintained, insecure versions of libraries. Then, when they're still on version 2.1.1, and your maintained version is 5.7.3, and somebody finds a major security bug in 2.1, they will come whining at you to release a 2.1.2.
Code that is not being maintained is not usually suitable for use, period.
Library maintainers have no right to police how people use their open source code, period. Maintainers are also not obligated to backport security fixes. Anything else is effectively against the concept of open source.
Notably, even this policing doesn’t fix the whining. The whining will just be about what TFA is whining about. You’re just moving the whining around.
It also does nothing to actually force people to upgrade. Instead, people can just cap against the minor version you broke your package on. Instead of being user hostile, why not make the user’s job easier?
Correctly following SemVer disincentivizes unnecessary breaking changes. That’s a very good thing for users and ultimately the health of the package. If you don’t want to backport security fixes, users are free to pay, do it themselves, or stop using the library.
> Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases.
I agree, but I think there's a bigger, cultural root cause here. This is the result of toxicity in the community.
The Python 2 to 3 transition was done properly, with real SemVer, and real tools to aid the transition. For a few years about 25% of my work as a Python dev was transitioning projects from 2 to 3. No project took more than 2 weeks (less than 40 hours of actual work), and most took a day.
And unfortunately, the Python team received a ton of hate (including threats) for it. As a natural reaction, it seems that they have a bit of PTSD, and since 3.0 they've been trying to trickle in the breaking changes instead of holding them for a 4.0 release.
I don't blame them--it's definitely a worse experience for Python users, but it's probably a better experience for the people working on Python to have the hate and threats trickle in at a manageable rate. I think the solution is for people like us who understand that breaking changes are necessary to pile love on doing it with real SemVer, and try to balance out the hate with support and
I had a client who in 2023 still was on 2.7.x, and when I found a few huge security holes in their code and told them I couldn't ethically continue to work on their product if they wouldn't upgrade Python, Django, and a few other packages, and they declined to renew my contract. As far as I know, they're still on 2.7.x. :shrug:
That’s probably true. I do think part of the anger is that a lot of the changes didn’t clearly improve the code around it. The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. More insidious was the breaks with strings and byte strings. It was a good and necessary change that also could lead to weird quiet breakages.
At least for me, the real blocker was broad package support.
Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
> The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit.
To be clear: I literally do not remember a single example of this breaking anything after running 2to3. There was some practical benefit (such as being able to use print in callbacks) and I don't think it breaking existing code is meaningful given how thoroughly automated the fix was.
I do get the impression that a lot of the complaints are from people who did not do any upgrades themselves, or if they did, didn't use the automated tools. This is just such an irrelevant critique. This is a quintessential example of bikeshedding: the only reason you're bringing up `print` is because you understand the change, not because it's actually important in any way.
> Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
Sure, but users in this case are blatantly wrong. You can read the discussions on each of the breaking changes, they're public in the PEPs. The dev team is obviously very concerned with causing downstream work for users, and made every effort, very successfully, to avoid such work.
If your impression is that maintainers didn't take into account downstream work for users, and your example is print, which frankly did not induce downstream work for users, you're the problem. You're being pretty disrespectful to people who put a lot of work into providing you a free interpreter.
I think we essentially agree. My comments about maintainers wasn’t referencing the Python language maintainers. The print change certainly shouldn’t have blocked anyone.
More interesting is how long it took core libraries to transition. That was my primary blocker. My guess is that there were fairly substantial changes to the CPython API that slowed that transition.
Other changes to strings could be actually dangerous if you were doing byte-level manipulations. Maybe tools could help catch those situations. Even if they did, it took some thought and not just find/replace to fix. The change was a net benefit, but it’s easy to see why people might be frustrated or delay transition.
Your definition of "core libraries" is likely a lot broader than mine. I'm old, and I remember back in the day when Perl developers started learning the hard way that CPAN isn't the Perl standard library.
JavaScript's culture has embraced pulling in libraries for every single little thing, which has resulted in stuff like the left pad debacle, but that very public failing is just the tip of the iceberg for what problems occur when you pull in a lot of bleeding edge libraries. The biggest problems, IMO, are with security. These problems are less common in Python's culture, but still fairly common.
I've come onto a number of projects to help them clean up codebases where development had become slow due to poor code quality, and the #1 problem I see is too many libraries. Libraries don't reduce complexity, they offload it onto the library maintainers, and if those library maintainers don't do a good job, it's worse than writing the code yourself. And it's not necessarily library maintainers' fault they don't do a good job: if they stop getting paid to maintain the library, or never were paid to maintain it in the first place, why should they do a good job of maintaining it?
The Python 2 to 3 transition wasn't harder for most core libraries than it was for any of the rest of us: if anything, it was easier for them because if they're a core library they don't have as many dependencies to wait on.
There are exceptions, I'm sure, but I'll tell you that Django, Pillow, Requests, BeautifulSoup, and pretty much every other library I use regularly, supported both Python 2 AND 3 before I even found out that Python 3 was going to have significant breaking changes. On the flip side, many libraries I had to upgrade had been straight up abandoned, and never transitioned from 2 to 3 (a disproportionate number of these were OAuth libraries, for some reason). I take some pride in the fact that most of the libraries that had problems with the upgrade were ones that had been imported when I wasn't at the company, or ones that I had fought against importing because I was worried about whether they would be maintained. It's shocking how many of these libraries were fixable not with an upgrade, but with removing the dependency writing a <100 lines of my own code including tests.
I'd hope the lesson we can take away from this isn't, "don't let Python make any breaking changes", but instead, "don't import libraries off Pypi just to avoid writing 25 lines of your own code".
The core libraries to me include all the numerical and other major scientific computing libraries. I’m guessing those were laggards due to things like that string/byte change and probably changed to the CPython API.
Did you ever look into why the transition took so long for OAuth libraries? Did you consider just rewriting one yourself?
Ah, I'm not so aware of the numerical/scientific computing space beyond numpy--I will say the numpy transition was pretty quick, though.
I did take the approach of writing my own OAuth using `requests`, which worked well, but I don't think I ever wrote in such a general way to make it a library.
Part of the problem is that OAuth isn't really a standard[1]. There are well-maintained libraries for Facebook and Google OAuth, but that's basically it--everyone else's OAuth is following the standard, but the standard is too vague so they're not actually compatible with each other. You end up hacking enough stuff around the library that it's easier to just write the thing yourself.
The problem with the Google and Facebook OAuth libraries is that there were a bunch of them--I don't think any one of them really became popular enough to become "the standard". When Python 3 came out, there were a bunch of new Google and Facebook OAuth libraries that popped up. I did actually port one Facebook OAuth library to Python3 and maintain it briefly, but the client dropped support for Facebook logins because too few users were using it, and Facebook kept changing data usage requirements. When the client stopped needing the library, I stopped maintaining it. It was on Github publicly, but as far as I know I was the only user, and eventually when I deleted the Repo nobody complained.
I don't say anything unless asked, but if asked I always recommend against OAuth unless you're using it internally: why give your sign up data to Google or Facebook? That's some of your most valuable data.
> It is important to know that NumPy, like Python itself and most other well known scientific Python projects, does not use semantic versioning. Instead, backwards incompatible API changes require deprecation warnings for at least two releases.
I don’t see why Valve wouldn’t try to support lots of hardware. Small time outfits like CachyOS can do it, why wouldn’t they? I think their motive with the Steam Box is hardware sales. It seems like they’re trying to shift the gaming ecosystem away from Windows.
Probably for the same reason they don't support every Arch package on the Steam Deck out-of-the-box; it breaks easily and it's not their job to fix it.
Additionally, I think Valve doesn't want to end up over-committed to replacing Windows. They can handle the storefront side and do a decent job with handling the runtime, but actually committing to a desktop alternative to Windows would be spreading their resources thin. It feels like a smart call to not jump into that arena if your hardware products don't need it.
Lots of assumptions that could be totally wrong. Custom distros are not new operating system - majority of hw support is because linux supports it. It would probably be more work for Valve to support only their hw instead of keeping what linux/arch support.
I am also not sure why you think they wouldn't want to end up replacing Windows. Thats exactly what they've been trying to do since microsoft included their store in windows. That is more than a decade long plan in motion which already failed once but they are still at it.
They're not assumptions, I outlined in my comment it was my thoughts and beliefs.
> Custom distros are not new operating system
Nothing is. Windows uses old DOS code, macOS uses BSD code, nobody's OS is truly written "from scratch" in 2025. Just because you can recycle old programs doesn't mean writing an OS is easy.
> majority of hw support is because linux supports it. It would probably be more work for Valve to support only their hw
And much of Linux hardware support is not in-kernel, period. Valve could not flip a switch and start supporting Asahi Macs or Nvidia's proprietary UNIX drivers; they would be committing to patching and maintaining all of their future quirks and surprises. Not even Valve should be wasting their time doing that.
> I am also not sure why you think they wouldn't want to end up replacing Windows.
They do! But "wanting to replace Windows" and "wanting to write the replacement for Windows" are two different things. Valve's current software team has a headcount lower than 500, they aren't equipped to compete with Microsoft even if they wanted to. It's much easier for them to ship all-in-one style devices that keep expectations low and replicate Windows' most desirable features.
> which already failed once
Steam Machine was a home console, it did not replace Windows for anything that wasn't directly ported to Linux. The lesson from this era is simple; supporting Linux is hard. It's hard for developers, hard for consumers and especially hard for Valve.
I guess “this weight class” is some theoretical class divorced from any application? Almost all players are running Nvidia other than Google. The other players are certainly more than just competing with Google.
> Almost all players are running Nvidia other than Google.
No surprises there, Google is not the greatest company at productizing their tech for external consumption.
> The other players are certainly more than just competing with Google.
TBF, its easy to stay in the game when you're flush with cash, and for the past N-quarters, investors have been throwing money at AI companies, Nvidia's margins have greatly benefited from this largesse. There will be blood on the floor once investors start demanding returns to their investments.
Ok? The person I was replying to was saying that Google’s compute offering is substantially superior to Nvidia’s. What do your comments about market positioning have to do with that?
If Google’s TPUs were really substantially superior, don’t you think that would result in at least short term market advantages for Gemini? Where are they?
They are suggesting it is easier for others to buy buy more NVidia chips and feed them more power. Whilst operating costs are covered by investors. If they move on to competing on having to do inference the cheepest then the TPUs will shine.
The original post made no comments about inference or training or even cost in any way. It said you could hook up more TPUs together with more memory and higher average bandwidth than you could with a datacenter of Nvidia GPUs. From an architectural point of view, it isn’t clear (and is not explained) what that enables. It clearly hasn’t led to a business outcome for Google where they are the clear market leader.
Seemingly fast interconnects benefit training more than inference since training can have more parallel communication between nodes. Inference for users is more embarrassingly parallel (requires less communication) than updating and merging network weights.
My point: cool benchmark, what does it matter? The original post says Nvidia doesn’t have anything to compete with massively interconnected TPUs. It didn’t merely say Google’s TPUs were better. It said that Nvidia can’t compete. That’s clearly bullshit and wishful thinking, right? There is no evidence in the market to support that, and no actual technical points have been presented in this thread either. OpenAI, Anthropic, etc are certainly competing with Google, right?
And then people explained why the effects are smoothed over right now but will matter eventually and you rejected them as if they didn't understand your question. They answered it, take the answer.
> It didn’t merely say Google’s TPUs were better. It said that Nvidia can’t compete.
Can't compete at clusters of a certain size. The argument is that anyone on nVidia simply isn't building clusters that big.
That’s missing the point. Things like tensor cores were added in parallel with improvements to existing computer and CUDA kernels from 10 years ago generally run without modification. Hardware architecture may change, but Nvidia has largely avoided changing how you interact with it.
They don't have to, CUDA is a high-level API in this respect. The hardware will conform to the demands of the market and the software will support whatever the compute capability defines, Nvidia is clearer than most about this.
Sure, but you do have to do it pretty quick. Let’s pick a H100. You’ve probably heard that just writing scalar code is leaving 90+% of the flops idle. But even past that, if you’re using the tensor core but using the wrong instructions you’re basically capped at 300-400 TFLOPS of the 1000 the hardware supports. If using the new instructions but poorly you’re probably not going to hit even 500 TFLOPS. That’s just barely better than the previous generation you paid a bunch of money to replace.
Portable doesn't normally mean that it runs on arbitrarily old hardware. CUDA was never portable, it only runs on Nvidia hardware. The question is whether old versions of Whisper GPU run on newer hardware, that'd be backwards compatibility.
The most mind boggling number in this article to me was PeopleSoft claiming it would cost $500 million to make a payroll system for the Canadian government. That’s thousands of working years of software developers. It’s such a huge scale that it seems pretty clear the project should never start. PeopleSoft should have been dumped and the project’s scope massively reevaluated.
> R failed to evolve, let's be honest. Python won via jupyter
Ju = Julia
Pyt = Python
Er = R
R is not only supported in Jupyter, it was there from the start. I’ve never written a single line of R. It is bizarre how little people know about their tools.
reply