We have just entered the 'Age of Information', where importance of data/information handling and processing is becoming (far) more important than traditional economy (based on coal and oil) and data/information processing structures have become global. But we are still only beginning to understand the behavior and mechanics of this global 'Information Age' and rules guiding its behaviors, thus also what is needed to make it to produce more Value for the whole Humanity. Development of programming infrastructure is based on the often repeated mantra "Release often, release quick", producing bloatware with enormous number of submodules which nobody can understand and control and which just waste electricity executing useless CPU cycles. We have overproduction of IT Tools, but underproduction of useful, working programs and developers/programmers are not able to perform better because of too frequent changes in their tools. Here is considered current situation of popular programming language Python and proposed some measures for improvement.
With exhaustion of Earth's natural resources, we have to behave better. Natural resources are replaced
with data – we have to squeeze more from efficiency and innovation. Worldwide spending on
Information Technology (IT) is constantly growing, research indicates that investments to IT increase
both return on assets (ROA) and return on equity (ROE) [
With growth of the IT has grown also its use of electricity, electricity consumption and carbon
impact of AI doubles in every 100 days [
The Artificial General Intelligence (a concept which still does not have exact formal definition) [
Modern world is based on worldwide ecosystem of programs and communication. Programming
(mostly) means using some programming language. Currently (mid-2024) the most popular and
widely used languages according to the Tiobe [
Java and Python are very different languages, but they appeared at the same time (1995-1996),
they were used to develop the first version of the Google search engine [
Java is a strongly typed object-oriented compiled language. The main idea of Java was to make
Java programs universally applicable - write once, run anywhere - compiled Java bytecode can be
executed without recompiling on all hardware that supports Java, i.e. for which has been developed
Java virtual machine (JVM). After acquisition of Sun Microsystems in 2009 by Oracle, Oracle is
presenting itself as the steward of Java technology, but the Java standard is controlled by the the Java
Community Process [
System.out.println("Hello World!"); }
}
Python is an interpreted script language invented at the CWI (Centrum Wiskunde & Informatica)
in the Netherlands by Dutch programmer Guido van Rossum. He needed tools for managing the
Amoeba operating system, so he designed a simple interpreted script language. The main attraction
of the Python language is its simplicity, minimal number of separators, simple visual indication of
scoop (by indentation), object-oriented style (no goto), variables do not have fixed type and are not
declared etc. Python development is coordinated by the Python Steering Council, which currently
has 5 members [
In Python the above “Hello World!” program could be expressed with only one line: print (“Hello World!”) 3.1. Reuse
All programs are reusing pieces of programs generated earlier by somebody. Authors of these pieces are claiming their authority, ownership with copyright, but already from the very beginning of the ‘’Software age” many people stated, that their programs should be free, available to everyone.
All programming languages use these pieces of free programs as library modules and library packages (groups of modules).
Use of libraries is growing in all languages (nobody wants to ‘re-invent bicycle’). Libraries call
other libraries thus they add enormous number of LOC (Lines Of Code) to the text of the program.
For instance, the diagram in Figure 2 is created by 10 LOC of Python 3.11.2 code. This code imports
one library pyplot, but pyplot imports many other libraries. All together are imported 768 libraries
which together contain 483056 LOC (and 113 bad libraries which were not found or give error on
opening), thus ‘the index of visibility’ [
Programming was once considered as an activity of writing down programming language’s commands in proper order. Nowadays programming is mostly importing modules and packages and for many programmers (especially students) loading libraries seem to be the main/only task in creating a new program. Before even analyzing what to do, how to solve a problem are loaded all modules what somehow seem to be related. Libraries contain many files, e.g. the popular Python library numpy (numeric Python) contains (under Python 3.11) 1426 files, thus nobody knows exactly, what there is, why it is there, how it works or does it work at all.
Programming languages and their libraries are frequently updated. Updates not only add new
features, but also remove or modify existing ones, e.g. in PEP (Python Enhancement Proposal) 594
[
But some modules may require just these removed ones, thus it may never be possible to have for all modules the latest versions. For instance, Python installer pip command to list outdated modules shows three modules:
tensorflow-intel 2.16.1 requires ml-dtypes~=0.3.1, but you have mldtypes 0.4.0 which is incompatible.
tensorflow-intel 2.16.1 requires tensorboard<2.17,>=2.16, but you have tensorboard 2.17.0 which is incompatible.
Successfully installed protobuf-4.25.3 tensorboard-2.17.0
pip list -o Package Version Latest Type -------- ------- ------ ----protobuf 4.25.3 5.27.1 wheel
Use of old modules by some other modules may cause errors and make these modules useless.
The programming infrastructure is drowning in updates. The Java and Python languages have together been updated during their existence in 1995-2024 altogether 795 times. For comparison – the ECMAScript (the base for Javascript, i.e. the whole web-page design) has traditionally only one update per year (in June); in the same period June 1997-June 2023 it had only 14 updates [19].
The calculated above third-order model predicts, that in 2025 will Python be updated 28 times (an update in less than two weeks) and Java – 22 times (an update in two and half weeks). How could a programming project manager or a teacher in university manage this flow? And the flow is accelerating, these models predict that in 2030 both languages will be updated 49 times, i.e. once in a week.
Java is used on server side, on network and there updates are rarely breaking already existing systems – this would cause lot of problems. Java updates replace the previous version and do not create a new language.
Python users are hobbyists, scientists, students and Python updates are not only for adding new features, the whole language gets a new major version number, becomes essentially a new language. For a major version exist several micro-versions, e.g. currently 5 versions for Python 3.12, 10 versions for Python 3.11 etc.; of those 5 (versions 3.8..3.12) are under development, thus for them will appear new releases (versions). From the Python language created by Guido van Rossum have been derived 26 new languages.
Why are at the same time developed five versions of the same product and why three of their five developers are developing at the same time two different versions of the Python language? Different versions were even released at the same day, e.g. versions 1.6, 2.0 and 2.6, 3.0 were released simultaneously.
It is also difficult to understand, how dates and tempo of next releases can be decided for years ahead. The Python “Development Cycle” [23] states:
“new major versions … only come when strongly incompatible changes are deemed necessary, and are planned very long in advance”
How can Python developers know for years ahead, when changes will “deemed necessary”? The PEP 361 from 29-Jun-2006 [24] projected 21 releases for years 2008-2013, in PEP 602 from 04-Jun2019 [25] “describes a change in the release calendar for Python …such that feature versions are released predictably every twelve months”. It seems that the holy goal is to fill the whole world with numerous versions of the Python language.
The philosophy of big number of releases was popularized by Eric S. Raymond in his 1997 essay "The Cathedral and the Bazaar" [26]. Raymond stated "Release early. Release often."
Raymond was at the time developer of the open-source mail project fetchmail, i.e. a software product that is directly used by customers. For developers of user programs quick releases are logical – users can benefit from new features. But for developers of tools (e.g. programming languages) frequent changes disrupt use of these tool, development of programs for users.
Raymond also added “And listen to your customers". Customers voice is definitely the most important feature showing software usability. Python developers consider as index of usability of a package the number of downloads [27]: “PyPI download statistics are the most powerful tool a Python developer can use to inform their decision making” [28].
This is a dubious idea: people download just to try out; if they got something useful and working they do not need download again (and also being afraid that next downloads/updates will break already working code). So the big number of downloads of a package may indicate the opposite – users are trying it, but could not get it to work. We tried out the most downloaded packages boto3, botocore (SDK for Amazon Web Services) in PyPI (Python Package Index) repository. We tried these packages in several computers (several downloads), but could not get it to work. We will try again after some time and thus increase the number of downloads. PyPI considers increase of downloads as indicator of the value of the package. But we could not use packages, thus actually increase of the number of dowloads may indicate that the package is not good.
New Python modules (often) uninstall some old ones and replace them with older versions, e.g. the updating tensorflow to 2.16.1 replaced (besides downgrading the tensorboard and protobuf modules) also the numpy 1.24.4 with older version 1.24.3. But these downgraded modules may also want to downgrade something else, thus soon it becomes impossible to say, what works and what does not work, the only way is to try out.
Most modules and packages do support earlier Python versions, but not farther than four versions back; Python also does not fix bugs further than 3-4 versions back. Thus it has become normal to have in computer installed several versions of Python, e.g. in our test computers are installed respectively Python 3.6.8, 3.7.4, 3.8.0, 3.12.3 (under Win64) and 2.7.18 under Windows 7; there is also a computer with Python 3.11.2 under 64-bit Windows 10. The py-loader can select a version Python, e.g. command “py -3.11” loads Python version 3.11.
Every Python language version has its own runtime and library (Python versions can use only their own libraries). Thus the same package gets installed on the same hard disk several times, e.g. the numpy package is in our development computer installed 5 times.
Some packages can be used only with some definite version of the Python language. The tensorflow package needs for better speed support of gpu - the graphics processor. For use of gpu is needed the CUDA toolkit from NVIDIA [29]. This toolkit depends on the gpu on the graphics card and when the next tensorflow version updated their requirements for gpu the tensorfow package was not working any more. We can not buy a new graphics card for every update of some package.
The py installer can be used in a small script to update all packages/modules under given version of the Python language. Updating all packages updated also tensorfow, but the new version requires higher graphics card, thus tensorfow become again useless.
To see what is installed in libraries of all installed Python versions we created a small program which traces all .py files in a library and counts number of files, which could be read without errors (good modules), total number of LOC in these files and number of files, which create errors on opening (bad modules).
There have been made many claims about increasing Python speed (“Python 3.11, 10–60% Faster than 3.10” [30], “Python 3.12: Faster, leaner…” [31]). It seems that authors of these claims consider some “ideal” computer, where cpu works without any distractions. But cpu works under Operating System (OS – Windows, MacOS, Linux etc) and can run as quickly as OS allows and use just as much memory as OS gives. Nowadays computers are connected to Internet and OS-s have all the time something to do. At the time of writing this the Tcpview [32] tool shows that several services (and the WinWord program itself) are constantly exchanging information with over 20 hosts. Speaking about increase of speed of Python e.g. under Windows is a very dubious enterprise; besides, Python can use only one core of modern multi-core cpu-s. To investigate we started a program which collects good and bad modules consequtively three times and measured the execution times.
The test indicated that in every Python version there were 1-2% of modules, which give errors on opening (TypeError, UnicodeDecodeError, ValueError, EOFError etc); the text representations of Python3.11.2 library modules contained 5551 errors.
Python libraries contain the program test___all__.py. Contrary to the name this program responds to errors with pass and skips (in Python 3.11) 1447 modules which do not have variable __all__ - why are these untested modules included in releases ?
Measuring the execution times shows, that times of consecutive executions differ largely in all Python versions. Execution time is essentially reduced by cashe – the function test___all__.py was on second execution nearly 5 times quicker.
Bad modules are present in all Python versions and are called also from working programs. We tested the small 10-lines program it_spending.py that creates the chart in Figure 6.
This test shows that the number of bad modules is greater in newer Python versions (17.7% in Python 3.12.3) and it is much lower in older versions (8.2% in Python 2.7.18).
This again supports the old wisdom “Don’t use the latest update” and makes odd Python developers calling language’s end of updates “end-of-life” – for all other programming languages these versions are called “stable” and instead “end-of-life” is used “start-of-use”.
The last test indicates that old versions of modules and packages may preferable to the big new ones. The matplotlib.pyplot package from all Python versions what we tested draw the chart in Figure 2 without errors. But the matplotlib package version 2.5 in Python 2.7.18 contains 557 files, 540 modules (8% bad); matplotlib version 3.9 under Python 3.12 contains 779 files, 660 modules (17.1% bad – over twice more), therefore the older versions (especially 2.7) are still widely used.
Thus it seems reasonable to suggest that Python language versions (installed in the same computer) should be able to share packages. Adding to the py-loader ability to load modules from libraries of other versions should not be very complex (updates have modified only Python runtime, the compiler has remained mostly the same) and this would make different versions of Python to look again as one language and not as 26 different languages.
The last two figures show that Python developers announcement “sunsetting Python 2” [33] was premature.
The Python language has 26 versions (so far) and lot of libraries which in turn also have many versions, e.g. the boto3 module has 2104 versions/updates [34]. In order to use the language user should install several versions; which all have their own libraries. Python language versions cannot use libraries of another version, although modules from older versions of the language contain fewer bad modules. Python modules cannot be shared by different versions of Python, and Python cannot share different versions of a package using the same runtime. Python user’s hard disk is full of dysfunctional and unsupported modules. Does something work or not can be discovered only by trying; this is a waste of electricity and developer’s time and mental energy and the ‘broken’ state of Python may be the reason why in April 2024 Google fired its entire Python team [35].
Python developers should substantially reduce number of releases of new versions of the Python language and remove from PyPI broken modules. The Python installer pip should not be allowed to uninstall from user’s computer some already installed modules and replace them with older versions what is the normal behavior of Microsoft installers. [19] 19 Webreference, A Brief History of ECMAScript Versions in JavaScript, 2024. URL: https://webreference.com/javascript/basics/versions/ . [20] 20 Java, JDK releases, 2024. URL: https://www.java.com/releases/. [21] 21 Python Software Foundation, Python Documentation by Version, 2024. URL: https://python.org/doc/versions/ . [22] 22 Python Software Foundation, Downloads, 2024. URL: https://www.python.org/ downloads/ . [23] 23 Python Developer’s Guide,2024. URL: https://devguide.python.org/developerworkflow/development-cycle/. [24] 24 Python Enhancement Proposals, PEP 361 – Python 2.6 and 3.0 Release Schedule, 2024. URL: https://peps.python.org/pep-0361/ . [25] 25 Python Enhancement Proposals, PEP 602 – Annual Release Cycle for Python, 2024. URL: https://peps.python.org/pep-0602/ . [26] 26 E.S. Raymond. The Cathedral & the Bazaar, O'ReiIly Media, Inc. 2001, ISBN:9780596001087. [27] 27 PyPI Stats, Most downloaded PyPI packages, 2024. URL: https://pypistats.org/top . [28] 28 Langui.Sh. Data Driven Decisions Using PyPI Download Statistics, 2016. URL: https://langui.sh/2016/12/09/data-driven-decisions/ . [29] 29Nvidia Developer, Your GPU Compute Capability, 2024. URL: ttps://developer.nvidia.com/ cuda-gpus#compute. [30] 30 Medium, Python 3.11, 10-60% Faster than 3.10., 2024. URL: https://medium.com/geekculture/python-3-11-10-60-faster-than-3-10-192acb4179f1. [31] 31 Infoworld, Python 3.12: Faster, leaner, more future-proof, 2024. URL: https://www.infoworld.com/article/3694512/python-312-faster-leaner-more-future-proof.html. [32] 32 Microsoft, TCPView v4.19, 2024. URL: https://learn.microsoft.com/en-us/sysinternals/ downloads/tcpview. [33] 33 Python Software3 Foundation, Sunsetting Python 2. URL: https://www.python.org/doc/ sunset-python-2/. [34] 34 Pypi.org, The AWS SDK for Python, Releaase history, boto3 1.34.129, 2024. URL: https://pypi.Org/project/boto3/#history. [35] 35 P. Krill. Google Lays off Python team. InfoWorld Apr 29, 2024, https://www.infoworld.com/article/2336932/google-lays-off-python-team-reports.html.