TL;DR
It's difficult to find the right Debian image. We have thousands of ISO files and cloud images and we support multiple CPU architectures and several download methods. The directory structure of our main image server is like a maze, and our web pages for downloading are also confusing.
Most important facts from this blog post
- https://cdimage.debian.org/cdimage/
- The newest netinst ISO
- Newest Debian live testing with GNOME
- There's no official Debian stable ISO using the backports kernel
- The FAIme web service creates custom images using Debian stable with a newer kernel
- The oldest image: Other-Maybe-Official - Debian-i386 2.1 Disc 1.iso
The Debian maze
Did you ever searched for a specific Debian image which was not the default netinst ISO for amd64? How long did it take to find it?
Debian is very good at hiding their images for downloading by offering a huge amount of different versions and variants of images and multiple methods how to download them. Debian also has multiple web pages for
- Download Debian
- Installing Debian via the Internet
- Debian installation media for USB, CD, DVD
- Network install from a minimal USB,CD
- Live install images
- Downloading Debian USB/CD/DVD images via HTTP/FTP
- Downloading Debian USB/CD/DVD images with BitTorrent
- Downloading Debian USB/CD/DVD images with jigdo
- Installing with the Debian-Installer
This is the secret Debian maze of images. It's currently filled with 8700+ different ISO images and another 34.000+ files (raw and qcow2) for the cloud images.
The main URL for the server hosting all Debian images is https://cdimage.debian.org/cdimage/
There, you will find installer images, live images, cloud images.
Let's try to find the right image you need
We have three different types of images:
- Installer images can be booted on a computer without any OS and then the Debian installer can be started to perform a Debian installation
- Live images boot a Debian desktop without installing anything to the local disks. You can give Debian a try and if you like it you can use the Calamers graphical installer for installing the same desktop onto the local disk.
- Cloud images are meant for running a virtual machine with Debian using QEMU, KVM, OpenStack or in the Amazon AWS cloud or Microsoft Azure cloud.
Images for the stable release
Almost always, you are probably looking for the image to install the latest stable release. The URL https://cdimage.debian.org/cdimage/release/ shows:
12.9.0
12.9.0-live
current
current-live
but you cannot see that two are symlinks:
current -> 12.9.0/
current-live -> 12.9.0-live/
Here you will find the installer images and live images for the stable release (currently Debian 12, bookworm).
If you choose https://cdimage.debian.org/cdimage/release/12.9.0/ you will see a list of CPU architectures:
amd64
arm64
armel
armhf
i386
mips64el
mipsel
ppc64el
s390x
source
trace
(BTW source and trace are no CPU architectures)
The typical end user will not care about most architectures, because your computer will actually always need images from the amd64 folder. Maybe you have heard that your computer has a 64bit CPU and even if you have an Intel processor we call this architecture amd64.
Let's see what's in the folder amd64:
bt-bd
bt-cd
bt-dvd
iso-bd
iso-cd
iso-dvd
jigdo-16G
jigdo-bd
jigdo-cd
jigdo-dlbd
jigdo-dvd
list-16G
list-bd
list-cd
list-dlbd
list-dvd
Wow. This is confusing and there's no description what all those folders mean.
- bt = BitTorrent, a peer-to-peer file sharing protocol
- iso = directories containing ISO files
- jigdo = a very special download option only for experts who know they really want this
- list = contains lists of the names of the .deb files which are included on the images
The first three are different methods how to download an image. Use iso when a single network connection will be fast enough for you. Using bt can result in a faster download, because it downloads via a peer-to-peer file sharing protocol. You need an additional torrent program for downloading.
Then we have these variants:
- bd = Blu-ray disc (size up to 8GB)
- cd = CD image (size up to 700MB)
- dvd = DVD images (size up to 4.7GB)
- 16G = for an USB stick of 16GB or larger
- dlbd = dual layer Blu-ray disc
16G and dlbd images are only available via jigdo.
All iso-xx and bt-xx folders provide the same images but with a
different access method.
Here are examples of images:
iso-cd/debian-12.9.0-amd64-netinst.iso
iso-cd/debian-edu-12.9.0-amd64-netinst.iso
iso-cd/debian-mac-12.9.0-amd64-netinst.iso
Fortunately the folder explains in very detail the differences between
these images and what you also find there.
You can ignore the SHA... files if you do not know what they are needed for.
They are not important for you.
These ISO files are small and contain only the core Debian installer
code and a small set of programs. If you install a desktop
environment, the other packages will be downloaded at the end of the installation.
The folders bt-dvd and iso-dvd only contain
debian-12.9.0-amd64-DVD-1.iso or the appropriate torrent file.
In bt-bd and iso-bd you will only find debian-edu-12.9.0-amd64-BD-1.iso.
These large images contain much more Debian packages, so you will not
need a network connection during the installation.
For the other CPU architectures (other than amd64) Debian provides less variants of images but still a lot. In total, we have 44 ISO files (or torrents) for the current release of the Debian installer for all architectures. When using jigdo you can choose between 268 images.
And these are only the installer images for the stable release, no older or newer version are counted here.
Take a breath before we're diving into.....
The live images
The live images in release/12.9.0-live/amd64/iso-hybrid/ are only available for the
amd64 architecture but for newer Debian releases there will be images also
for arm64.
We have 7 different live images containing one of the most common desktop environments and one with only a text interface (standard).
debian-live-12.9.0-amd64-xfce.iso
debian-live-12.9.0-amd64-mate.iso
debian-live-12.9.0-amd64-lxqt.iso
debian-live-12.9.0-amd64-gnome.iso
debian-live-12.9.0-amd64-lxde.iso
debian-live-12.9.0-amd64-standard.iso
debian-live-12.9.0-amd64-cinnamon.iso
debian-live-12.9.0-amd64-kde.iso
The folder name iso-hybrid is the technology that you can use those ISO files for
burning them onto a CD/DVD/BD or writing the same ISO file to a USB stick.
bt-hybrid will give you the torrent files for downloading the
same images using a torrent client program.
More recent installer and live images (aka testing)
For newer version of the images we have currently these folders:
daily-builds
weekly-builds
weekly-live-builds
trixie_di_alpha1
I suggest using the weekly-builds because in this folder you find
a similar structure and all variants of images as in the release
directory. For e.g.
weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
and similar for the live images
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-kde.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-lxde.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-debian-junior.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-standard.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-lxqt.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-mate.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-xfce.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-gnome.iso
weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-cinnamon.iso
weekly-live-builds/arm64/iso-hybrid/debian-live-testing-arm64-gnome.iso
Here you see a new variant call debian-junior, which is a Debian
blend. BitTorrent files are not available for weekly builds.
The daily-builds folder structure is different and only provide the small network
install (netinst) ISOs but several versions of the last
days. Currently we have 55 ISO files available there.
If you like to use the newest installation image fetch this one:
Debian stable with a backports kernel
Unfortunately Debian does not provide any installation media using the stable release but including a backports kernel for newer hardware. This is because our installer environment is a very complex mix of special tools (like anna) and special .udeb versions of packages.
But the FAIme web service of my FAI project can build a custom installation image using the backports kernel. Choose a desktop environment, a language and add some packages names if you like. Then select Debian 12 bookworm and then enable backports repository including newer kernel. After a short time you can download your own installation image.
Older releases
Usually you should not use older releases for a new installation. In our archive the folder https://cdimage.debian.org/cdimage/archive/ contains 6163 ISO files starting from Debian 3.0 (first release was in 2002) and including every point release.
The full DVD image for the oldstable release (Debian 11.11.0 including non-free firmware) is here
the smaller netinst image is
https://cdimage.debian.org/cdimage/archive/11.10.0/amd64/iso-cd/debian-11.10.0-amd64-netinst.iso
The oldest ISO I could find is from 1999 using kernel 2.0.36
I still didn't managed to boot it in KVM.
UPDATE I got a kernel panic because the VM had 4GB RAM. Reducing this to 500MB RAM (also 8MB works) started the installer of Debian 2.1 without any problems.
Anything else?
In this post, we still did not cover the ports folder (the non official supported (older) hardware architectures) which contains around 760 ISO files and the unofficial folder (1445 ISO files) which also provided the ISOs which included the non-free firmware blobs in the past.
Then, there are more than 34.000 cloud images. But hey, no ISO files are involved there. This may be part of a complete new posting.
The new years starts with a FAI release. FAI 6.2.5 is available and contains many small improvements. A new feature is that the command fai-cd can now create ISOs for the ARM64 architecture.
The FAIme service uses the newest FAI version and the Debian most recent point release 12.9. The FAI CD images were also updated. The Debian packages of FAI 6.2.5 are available for Debian stable (aka bookworm) via the FAI repository adding this line to sources.list:
deb https://fai-project.org/download bookworm koeln
Using the tool extrepo, you can also add the FAI repository to your host
# extrepo enable fai
FAI 6.2.5 will soon be available in Debian testing via the official Debian mirrors.
A Brief History of FAI, Which Began 25 Years Ago
On Dec 21st, 1999 version 1.0 of FAI (Fully Automatic Installation) was announced. That was 25 years ago.
Some months before, the computer science department of the University of Cologne bought a small HPC cluster with 16 nodes (each with dual CPU Pentium II 400Mhz, 256 MB RAM) and I was too lazy to install those nodes manually. That's why I started the FAI project. With FAI you can install computers in a few minutes from scratch to a machine with a custom configuration that is ready to go for their users.
At that time Debian 2.1 aka slink was using kernel 2.0.36 and it was the first release using apt. Many things have happened since then.
In the beginning we wrote the first technical report about FAI and a lot of documentation were added afterwards. I gave more than 45 talks about FAI all over the world. Over the past 25 years, there has been an average of more than one commit per day to the FAI software repository.
Several top500.org HPC clusters were built using FAI and many companies are using FAI for their IT infrastructure or deploying Linux on their products using FAI. An overview of users can be found here.
Some major milestones of FAI are listed in the blog post of the 20th anniversary.
What Happended in the Last 5 Years?
- Live images can be created
- Writeable data partition on USB sticks
- FAIme web service creates custom live ISOs
- Support for Alpine Linux and Arch Linux package managers
- Automatic detect a local config space
- Live and installation images for Debian for new hardware using a backports kernel or using the Debian testing release
- The FAIme web services created more than 30.000 customized ISOs
Currently, I'm preparing for the next FAI release and I still have ideas for new features.
Thanks for all the feedback from you, which helped a lot in making FAI a successful project.
About FAI
FAI is a tool for unattended mass deployment of Linux. It's a system to install and configure Linux systems and software packages on computers as well as virtual machines, from small labs to large-scale infrastructures like clusters and cloud environments. You can take one or more virgin PC's, turn on the power, and after a few minutes, the systems are installed, and completely configured to your exact needs, without any interaction necessary.
The topic of this posting already tells you that an old Unix guy tells stories about old techniques.
I'm a happy NIS (formerly YP) user since 30+ years. I started using it with SunOS 4.0, later using it with Solaris and with Linux since 1999.
In the past, a colleague wasn't happyly using NIS+ when he couldn't log in as root after a short time because of some well known bugs and wrong configs. NIS+ was also much slower than my NIS setup. I know organisations using NIS for more than 80.000 user accounts in 2024.
I know the security implications of NIS but I can live with them, because I manage all computers in the network that have access to the NIS maps. And NIS on Linux offers to use shadow maps, which are only accessible to the root account. My users are forced to use very long passwords.
Unfortunately NIS support for the PAM modules was removed in Debian in pam 1.4.0-13, which means Debian 12 (bookworm) is lacking NIS support in PAM, but otherwise it is still supported. This only affects changing the NIS password via passwd. You can still authenticate users and use other NIS maps.
But yppasswd is deprecated and you should not use it!
If you use yppasswd it may generate a new password hash by using the
old DES crypt algorithm, which is very weak and only uses the first 8
chars in your password. Do not use yppasswd any more!
yppasswd only detects DES, MD5, SHA256 and SHA512 hashes, but for me
and some colleagues it only creates weak DES hashes after a password
change. yescrypt hashes which are the default in Debian 12 are not
supported at all. The solution is to use the plain passwd program.
On the NIS master, you should setup your NIS configuration to use
/etc/shadow and /etc/passwd even if your other NIS maps are in
/var/yp/src or similar. Make sure to have these lines in your
/var/yp/Makefile:
PASSWD = /etc/passwd
SHADOW = /etc/shadow
Call make once, and it will generate the shadow and passwd map. You may want to set the variable MINUID which defines which entries are not put into the NIS maps.
On all NIS clients you still need the entries (for passwd, shadow, group,...) that point to the nis service. E.g.:
passwd: files nis systemd
group: files nis systemd
shadow: files nis
You can remove all occurences of "nis" in your /etc/pam.d/common-password file.
Then you can use the plain passwd program to change your password on
the NIS master. But this does not call make in /var/yp for updating
the NIS shadow map.
Let's use inotify(7) for that. First, create a small shell script
/usr/local/sbin/shadow-change:
#! /bin/sh
PATH=/usr/sbin:/usr/bin
# only watch the /etc/shadow file
if [ "$2" != "shadow" ]; then
exit 0
fi
cd /var/yp || exit 3
sleep 2
make
Then install the package incron.
# apt install incron
# echo root >> /etc/incron.allow
# incrontab -e
Add this line:
/etc IN_MOVED_TO /usr/local/sbin/shadow-change $@ $# $%
It's not possible to use IN_MODIFY or watch other events on /etc/shadow directly, because the passwd command creates a /etc/nshadow file, deletes /etc/shadow and then moves nshadow to shadow. inotify on a file does not work after the file was removed.
You can see the logs from incrond by using:
# journalctl _COMM=incrond
e.g.
Oct 01 12:21:56 kueppers incrond[6588]: starting service (version 0.5.12, built on Jan 27 2023 23:08:49)
Oct 01 13:43:55 kueppers incrond[6589]: table for user root created, loading
Oct 01 13:45:42 kueppers incrond[6589]: PATH (/etc) FILE (shadow) EVENT (IN_MOVED_TO)
Oct 01 13:45:42 kueppers incrond[6589]: (root) CMD ( /usr/local/sbin/shadow-change /etc shadow IN_MOVED_TO)
I've disabled the execution of yppasswd using dpkg-divert
# dpkg-divert --local --rename --divert /usr/bin/yppasswd-disable /usr/bin/yppasswd
chmod a-rwx /usr/bin/yppasswd-disable
Do not forget to limit the access to the shadow.byname map in
ypserv.conf and general access to NIS in ypserv.securenets.
I've also discovered the package pamtester, which is a nice package
for testing your pam configs.
The number of FAIme jobs has reached 30.000. Yeah!
At the end of this November the FAIme web service for building customized ISOs turns 7 years old.
It had reached 10.000 jobs in March 2021 and 20.000 jobs were reached in
June 2023. A nice increase of the usage.
Here are some statistics for the jobs processed in 2024:
Type of jobs
| 3% | cloud image | |
| 11% | live ISO | |
| 86% | install ISO |
Distribution
| 2% | bullseye | |
| 8% | trixie | |
| 12% | ubuntu 24.04 | 78% | bookworm |
Misc
- 18% used a custom postinst script
- 11% provided their ssh pub key for passwordless root login
- 50% of the jobs didn't included a desktop environment at all, the others used GNOME, XFCE or KDE or the Ubuntu desktop the most.
- The biggest ISO was a FAIme job which created a live ISO with a desktop and some additional packages This job took 30min to finish and the resulting ISO was 18G in size.
Execution Times
The cloud and live ISOs need more time for their creation because the FAIme server needs to unpack and install all packages. For the install ISO the packages are only downloaded. The amount of software packages also affects the build time. Every ISO is build in a VM on an old 6-core E5-1650 v2. Times given are calculated from the jobs of the past two weeks.
| Job type | Avg | Max | ||
| install no desktop | 1 min | 2 min | ||
| install GNOME | 2 min | 5 min |
The times for Ubuntu without and with desktop are one minute higher than those mentioned above.
| Job type | Avg | Max | ||
| live no desktop | 4 min | 6 min | ||
| live GNOME | 8 min | 11 min |
The times for cloud images are similar to live images.
A New Feature
For a few weeks now, the system has been showing the number of jobs ahead of you in the queue when you submit a job that cannot be processed immediately.
The Next Milestone
At the end of this years the FAI project will be 25 years old. If you have a success story of your FAI usage to share please post it to the linux-fai mailing list or send it to me. Do you know the FAI questionnaire ? A lot of reports are already available.
Here's an overview what happened in the past 20 years in the FAI project.
About FAIme
FAIme is the service for building your own customized ISO via a web interface. You can create an installation or live ISO or a cloud image. Several Debian releases can be selected and also Ubuntu server or Ubuntu desktop installation ISOs can be customized. Multiple options are available like selecting a desktop and the language, adding your own package list, choosing a partition layout, adding a user, choosing a backports kernel, adding a postinst script and some more.
At this years Debian conference in South Korea I've presented1 the new feature of the FAIme web service. You can now build your own Debian live media/ISO.
The web interface provides various settings, for e.g. adding a user name and its password, selecting the Debian release (stable or testing), the desktop environment and the language. Additionally you can add your own list of packages, that will be installed into the live environment. It's possible to define a custom script that gets executed during the boot process. For remote access to the live system, you can easily sepcify a github, gitlab or salsa account, whose public ssh key will be used for passwordless root access. If your hardware needs special grub settings, you may also add those. I'm thinking about adding an autologin checkbox, so the live media could be used for a kiosk system.
And finally newer hardware is supported with the help of the backports kernel for the Debian stable release (aka bookworm). This combination is not available from the official Debian live images or the netinst media because the later has some complicated dependencies which are not that easy to resolve2. At DebConf24 I've talked to Alper who has some ideas3 how to improve the Debian installer environment which then may support a backports kernel.
The FAI web service for live ISO is available at
Debian Download Web Page
It's just a very tiny difference, but hopefully a big step forward for our users. Our main download web page (which still uses the URL https://www.debian.org/distrib/) now has the title "Download Debian". Hopefully this will improve the results in the search engines.
A brief history of this web page in time
- 1998: The title "Distribution" was added
- 2002: Title changed to "Getting Debian"
- 2024: Finally changed to "Download Debian"
Here are the screenshots of these three versions.
I like that we had a selection menu on the top right corner to select a mirror for downloading in the past.
A few days ago I've also removed the info "Internal ISDN cards are unfortunately not supported." from the netinst subpage. Things are moving forward, but slowly.
In two weeks DebConf24, the Debian conference starts in Busan, South Korea. Therefore I've added support for the Korean language into the web service of FAI:
https://fai-project.org/FAIme/
Another new feature of the FAIme service will be announced at DebConf24 in August.
A new FAI version was released and the FAIme service is using this new release. You can now also create installation images for Debian 13 (testing aka Trixie).
https://fai-project.org/FAIme/
Another new feature of the FAIme service will be announced at DebConf24 in August.
In January I've removed tens of thousands of web pages on www.debian.org. Have you noticed it?
In the past
From 1997 onwards, we had web pages for security announcements. We had to manually prepare a .data and a .wml file which then generated a web page for each security announcement (DSA or DLA). We have listed the 6 most recent messages in a short list that was created from these files. Most of the work that went into the Debian web pages was creating these files.
Our search engine often listed the pages with security announcements instead of a more relevant web page for a particular topic.
Preparation
At DebConf Kosovo (2022) I started with a proof of concept and wrote a script, that generates this list without using the .data/.wml files in the Git repository, but instead reading the primary sources of security information[1]. This new list now includes links to the security tracker and the email of the announcement.
Following web pages and scripts were also using these .data and .wml files:
- OVAL files
- RSS feeds for security announcements (and LTS)
- Apache config file for mapping URLs from dsa-NNN to YEAR/dsa-NNN
- A huge list of crossreferences between DSA and CVE numbers
Before I could remove all the security web pages, I had to adjust the scripts, that create the above information.
When I looked at the OVAL files and the apache logs of our web server, I saw that more than 99% of the web traffic was generated by these XML files (134TB of 135TB total in two weeks). They were not compressed and were around 50MB in size. With the help of Carsten Schönert we managed to modify the python scripts that generate this OVAL file without using the .data/.wml files and now we only provide bzip2 compressed XML files[2].
The RSS feeds are created by the new Perl script which reads the DSA/DLA list the security tracker and determines the URL of the email of all entries. This script also generates the list of the most recent DSA/DLA entries. Currently we show the last 350 entries which covers more than the last year and includes links to the announcement email and the security tracker.
The huge list of crossreferences is not needed any more, since the mapping of CVE to DSA is already included in the DSA list[3] of the security tracker.
The amount of translations of the DSA/DLA was very different. French translations were almost all done, but all other languages did translations for a couple of months or years only. E.g. in 2022, Italian had 2 translations, Russian 15, Danish 212, French and English each 279. But from 2023 on only French translations were made. By generating the list of DSA/DLA we lost the ability to translate these web pages, but since these announcements are made of simple, identical sentences it is easy to use an automatic translation service if needed.
Now the translation statistics of all web pages are more accurate. Instead of 12200 pages that need to be translated (including all these old DSA/DLA) there are now only 2500 pages to translate[4]. Languages that had a lot of old translations of DSA/DLA lost some percentage but languages that are doing translations of newer web pages won in the statistics of how many pages are translated. Examples:
Before
German (de) 3501 28.5%
Italian (it) 1005 8.2%
Danish (da) 6336 51.7%
After
German (de) 1486 59.0%
Italian (it) 909 36.1%
Danish (da) 982 39.0%
Cleanup of all the security web pages
Finally in January, I could remove all web pages of the security announcements in
one git commit[5].
Using several git rm -rf commands this commit
removed 54335 files, including around 9650
DSA/DLA data files, 44189 wml files, nearly 500 Makefiles.
Outcome
No more manual work is needed for the security team and we now have direct links from a DSA-NNN/DLA-NNN to the email in our mailing list archive. This was not possible before. The search results became more accurate.
But we still host a lot of other old content on the Debian web pages which may be removed in the future.
[1] https://www.debian.org/security/#infos
[2] https://www.debian.org/security/oval/
[3] https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DSA/list
[4] https://www.debian.org/devel/website/stats
[5] https://salsa.debian.org/webmaster-team/webwml/-/commit/2aa73ff15bfc4eb2afd85c
This blog is powered by ikiwiki.