View Full Version : Python and sysadmins

2012-May-19, 10:25 AM
This is a follow on from a comment I made in a thread in Q&A about the Python programming language. All you snake fans switch off now.

I also ought to highlight that this is all based on my hugely biased opinion and my experiences with the language. It should therefore be ignored by all sane and reasonable people. I will also say up front: I have used Python, I like the idea of Python and I have developed some nifty tools in the language thanks largely to the things about it I am about to rant about. I have no problem with hypocrisy.

So why did I say that Python makes sysadmins weep in a development environment? Simple. Libraries. Most of the really good things you can do with it come from libraries developed by other people. So the first issue is keeping track of which libraries the developers have installed. That alone is a nightmare. But when you add on the fact that libraries are often written to a very low level of standards compliance (not even sure there is one agreed Python standard!) it becomes a nightmare of dependencies.

Then there is the backwards compatibility... Or lack of it. This is probably my biggest issue with it. Even minor increments often trash all your libraries. 2-> 3 was a genius example of that. But I have also had issues with certain libraries (TKInter being a classic one) on minor version increments.

These problems have led some software companies who use Python as an extensible scripting language to their tools (like ESRI used to) to install their own versions of Python with the tool itself. Then, thanks to the backwards compatibility issues, you end up having to rebuild half your work for each release. Or the release ends up using year old versions and you have to develop against multiple versions of Python. And keeping them all playing nice on your systems.

Those are the main reasons I said Python makes sysadmins cry. It is easy if you just have one dev team who use one agreed set of libraries and one version of Python. But as soon as you get several teams all doing their own thing it becomes a nightmare.

It is an amazingly powerful language but a really hard one to manage across multiple systems/teams/software platforms. And yes, I still have to use it. And my sysadmin still weeps quietly.

Nowhere Man
2012-May-19, 04:32 PM
The Python version of DLL Hell. Java has JAR Hell. The only language I know of that doesn't is (or at least was, I'm no longer current on it) is Delphi, where everything was compiled into the EXE file (unless you insisted on using a OCX component).


2012-May-19, 05:15 PM
I guess I have just found that C++, Java and so on tend to have better tested and more compliant libraries. Sure you get some classes and so on - but nowhere near as often as you do with Python. It really has made an impression on me as an anarchic language!

Celestial Mechanic
2012-May-20, 04:31 AM
Any possibility that the creators/maintainers of Python will come to their senses and set up or allow the setting up of a standards committee? Or will pigs have to fly first?

2012-May-20, 04:52 AM
It is improving - the core bundle is getting more organised. Part of the reason 3.x totally breaks 2.x is because of this. And as it is being used for more than just prototyping/experimental code I think there will be a push to get its act together. But given the richness of the libraries out there it could take a while. They have started making guides to making and distributing packages - with luck this will morph into an agreed standard.

2012-May-21, 02:07 AM
Interesting thoughts.

The 2->3 transition was very explicit about breaking the API. This was intentional, forewarned far in advance, and 2 will be supported with maintenance fixes for quite some time. So I'd put that in a completely different category than problems with other libraries. What problems did you have with python 2? I've transitioned from 2.4 through 2.7, and didn't find any particular backwards compatibility issues.

The packages I use most often are numpy, scipy, pyfits, and matplotlib. Of those, matplotlib is the most problematic when it comes to questionable API changes between versions. It's also probably the most fluid, with fairly regular updates and often bugfixes (and the occasional bug introduced). When I started using python ~7 years ago, matplotlib was almost unuseable, and numpy didn't really exist, so they've definitely come a long way since then. Scipy and numpy are pretty solid, though, these days.

You said you had problems with tkinter? What other packages did you think were particularly bad?

I guess I haven't really had much trouble with versioning, as most of what I want is in fink (www.finkproject.org/) on OS X, or is already packaged for Debian/Ubuntu. Those packages that aren't are usually small things. Hmmm... I wonder if I'm just forgetting bad things?

2012-May-21, 05:34 AM
In using 2 my biggest bugbears were the psychopg and tkinter packages. Scipy was a little twitchy at times - mainly down to clashes between versions of Numpy that were pre-installed with other Python packages. Pythonxy was a firm favourite of some developers but the version they had seemed to have something different about it that made the standard Numpy/Scipy packages twitch at unpredictable times. But I would agree that they are among the most robust Python libraries out there. As far as apps go ArcGIS, Django and the afore mentioned Pythonxy seemed to each want their own private install of Python in order to not die horribly. As you can imagine having up to 5 versions of the same language of your system, some of which you can update, some you cannot, made for a painful tangle of code.

Actually matplotlib was one of the things that made me assault the desktop with my head on a regular basis (and I am not a sysadmin). It seemed to be very dependent on the version of Python used and switching from 2.4 to 2.6 seemed to confuse the 2.4 version no end. That just shouldn't happen! And trying to line up the right versions of half a dozen libraries, when some were not updated quickly, meant every minor version upgrade was a major logistical nightmare. I am sure this will improve with time though, it certainly has got better in the last few years.

While the 2.7 version will be maintained it will not be updated - they have said that pretty explicitly. So in the end it is going to mean a complete rebuild and test of anything written in it. I know they warned people but it does change the fact that as a business solution Python became much less attractive the moment they did that.

2012-May-21, 10:31 AM
Perl suffers from exactly the same problem.

Basically I think this is a problem common to all languages with a strong culture of community developed modules/libraries and the resulting large library/module portfolio.

Sturgeons Law hits hard when you have everyone and their grandma trying to make THE module that solves a common problem, resulting in multiple solutions, all of varying quality, varying support and no way to figure out which are the ones that are going to last.

2012-May-21, 03:57 PM
Basically I think this is a problem common to all languages with a strong culture of community developed modules/libraries and the resulting large library/module portfolio.
Indeed and I would hate to give people the impression that I think Python is the worst language for that - it is just the worst I have had to deal with. C/C++ and Java tend (at least in my experience) to be easier to trace dependencies in and be more backwards compatible releases.

There are some good packages out there, well maintained and looked after and more are joining their ranks all the time. SciPy/NumPy are pretty much there (but not so well managed as something like the NAG). There are some well managed geospatial libraries.