creating custom os

[maereax]

Hello, I'm looking for guidance on creating custom versions of raspbian with piserver. I want to PXE boot my pi's fully configured, one as a pihole, one for FlightRadar24, etc. I'm a linux noob and struggle with CLI work and basic linux directory structures.

So far my understanding of the process is that I need to configure a pi how I want it, and then run one of the scripts in the/var/lib/piserver/scripts directory and point it at my boot and root.tar files and then specify an output image to be copied to and added to the piserver

Is that accurate? and if so, could someone please point me towards the correct script(I'm assuming convert.sh but am not 100%) and the locations of the boot and root.tar files? Thank you for this awesome project, it's something I've been looking to do for a long time and was very excited when I found it.

[maxnet]

I want to PXE boot my pi's fully configured, one as a pihole, one for FlightRadar24, etc.

Not sure if that is going to work with piserver.
piserver was designed to be used for multi-user desktop environments (e.g. classrooms) and not to run server software.
The file system is read-only until a user logs in with a username and password. So your daemon software has no place to write data to.

If you do not need multi-user support, want all your Pi's to be running independent (not the same OS image), and only want your Pi's to store their data at a central location, Berryboot's iSCSI support may be a better fit for you: https://www.berryterminal.com/doku.php/storing_your_files_on_a_synology_nas_using_iscsi

==

But for educational purposes, here are the instructions for creating a custom piserver image:

the locations of the boot and root.tar files?

.tar.xz files are archives, a bit similar to .zip files.
You need to make those yourself first from the contents of your SD card.

A standard Raspbian SD card has two partitions. A FAT partition with the boot files. The contents of that should end up in boot.tar.xz
And an ext4 partition with the operating system itself. The contents of that should end up in root.tar.xz

1. Attach a SD card reader to the pc running piserver, and put the SD card you want to convert into it.
2. Open a terminal window.
3. Figure out the device name of the SD card is (e.g. sda). If you inserted (hotplugged) the SD card after the computer was already running, the easiest way is to enter "dmesg" and look at the last lines in the output.
4. run the following commands replacing sdX with the actual name of the device.

sudo bash
mkdir /tmp/mnt_boot /tmp/mnt_root
mount -o ro /dev/sdX1 /tmp/mnt_boot
mount -o ro /dev/sdX2 /tmp/mnt_root
cd /tmp/mnt_boot
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../boot.tar .
cd /tmp/mnt_root
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../root.tar .
cd /tmp
umount /tmp_mnt_boot
umount /tmp_mnt_root
xz boot.tar
xz root.tar
/var/lib/piserver/scripts/convert.sh boot.tar.xz root.tar.xz Your_custom_OS.tar.xz
rm boot.tar.xz root.tar.xz

5. The image you can import into piserver is now at /tmp/Your_custom_OS.tar.xz

[maereax]

Thanks for the info, you're right, this probably isn't the project for me, but this information is great and I appreciate it very much, thanks!

[FaberfoX]

I want to PXE boot my pi's fully configured, one as a pihole, one for FlightRadar24, etc.
...
But for educational purposes, here are the instructions for creating a custom piserver image:

the locations of the boot and root.tar files?

.tar.xz files are archives, a bit similar to .zip files.
You need to make those yourself first from the contents of your SD card.

sudo bash
mkdir /tmp/mnt_boot /tmp/mnt_root
mount -o ro /dev/sdX1 /tmp/mnt_boot
mount -o ro /dev/sdX2 /tmp/mnt_root
cd /tmp/mnt_boot
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../boot.tar .
cd /tmp/mnt_root
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../root.tar .
cd /tmp
umount /tmp_mnt_boot
umount /tmp_mnt_root
xz boot.tar
xz root.tar
/var/lib/piserver/scripts/convert.sh boot.tar.xz root.tar.xz Your_custom_OS.tar.xz
rm boot.tar.xz root.tar.xz

1. The image you can import into piserver is now at /tmp/Your_custom_OS.tar.xz

This is failing as of now as pxz is not available on buster.

[MilesAheadToo]

I want to PXE boot my pi's fully configured, one as a pihole, one for FlightRadar24, etc.

Not sure if that is going to work with piserver. piserver was designed to be used for multi-user desktop environments (e.g. classrooms) and not to run server software. The file system is read-only until a user logs in with a username and password. So your daemon software has no place to write data to.

If you do not need multi-user support, want all your Pi's to be running independent (not the same OS image), and only want your Pi's to store their data at a central location, Berryboot's iSCSI support may be a better fit for you: https://www.berryterminal.com/doku.php/storing_your_files_on_a_synology_nas_using_iscsi

==

But for educational purposes, here are the instructions for creating a custom piserver image:

the locations of the boot and root.tar files?

.tar.xz files are archives, a bit similar to .zip files. You need to make those yourself first from the contents of your SD card.

A standard Raspbian SD card has two partitions. A FAT partition with the boot files. The contents of that should end up in boot.tar.xz And an ext4 partition with the operating system itself. The contents of that should end up in root.tar.xz

1. Attach a SD card reader to the pc running piserver, and put the SD card you want to convert into it.
2. Open a terminal window.
3. Figure out the device name of the SD card is (e.g. sda). If you inserted (hotplugged) the SD card after the computer was already running, the easiest way is to enter "dmesg" and look at the last lines in the output.
4. run the following commands replacing sdX with the actual name of the device.

sudo bash
mkdir /tmp/mnt_boot /tmp/mnt_root
mount -o ro /dev/sdX1 /tmp/mnt_boot
mount -o ro /dev/sdX2 /tmp/mnt_root
cd /tmp/mnt_boot
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../boot.tar .
cd /tmp/mnt_root
bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../root.tar .
cd /tmp
umount /tmp_mnt_boot
umount /tmp_mnt_root
xz boot.tar
xz root.tar
/var/lib/piserver/scripts/convert.sh boot.tar.xz root.tar.xz Your_custom_OS.tar.xz
rm boot.tar.xz root.tar.xz

5. The image you can import into piserver is now at /tmp/Your_custom_OS.tar.xz

I think there is a typo in the instructions. I believe the unmount instructions should be:
umount /tmp/mnt_boot umount /tmp/mnt_root

Also I got the follow error message:
./var/lib/samba/private/msg.sock/676: gnutar format cannot archive sockets

I am using samba to connect to the RPi running PiServer to which 5 large drives are connected using a USB3 hub. The access to the 5 drives works perfectly on the RPi i'm using to build the image.

I haven't yet tried to use the generated image on PiServer.
Any suggestions?

[MilesAheadToo]

This is the full list of error messages I get when running bsdtar on root:

bsdtar --numeric-owner --format gnutar --one-file-system -cpf ../root.tar . : ./home/pi/.pcsc10/pcscd.comm: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/609: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/580: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/713: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/714: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/623: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/691: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/7573: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/7580: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/658: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/712: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/717: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/693: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/657: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/677: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/699: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/690: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/719: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/685: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/7579: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/718: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/652: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/701: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/610: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/637: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/555: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/613: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/711: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/716: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/572: gnutar format cannot archive sockets : ./var/lib/samba/private/msg.sock/676: gnutar format cannot archive sockets : ./tmp/.ICE-unix/1318: gnutar format cannot archive sockets : ./tmp/.ICE-unix/828: gnutar format cannot archive sockets : ./tmp/.X11-unix/X0: gnutar format cannot archive sockets

[maxnet]

This is the full list of error messages I get when running bsdtar on root:

Unix socket files cannot be in a .tar file.

I don't think you really need to have the Samba SERVER stuff running on the CLIENTS.
So you may want to "systemctl disable" smbd, nmbd, samba-ad-dc, etc., before making the image anyway.

[MilesAheadToo]

Good point about the samba SERVER stuff. I uninstalled all the samba stuff and then reinstalled smbclient. I then rebooted and the samba shares still get mounted. So I'll try again to make the img.

As an aside, stop the RPi, take the SD card and load it into a USB SD card reader, put it into the RPi running PiServer, and make the image from there. So the samba server stuff wouldn't have been running when I ran through the instructions.

Also, I have tried using the PiServer utility to create an image from an SD card, and it has always failed so far. I tried it on the latest SD card - with samba removed - and it failed.

[MilesAheadToo]

I got the same message about sockets.

I may try creating the SD from scratch again, and stop just before I install samba. Hopefully that will solve the problem.

I start from the RaspOS Lite and then run the following:

sudo apt update -y
sudo apt upgrade -y
sudo apt install -y xserver-xorg xfce4 xfce4-goodies xrdp gparted
sudo dpkg-reconfigure lightdm

Do you see anything that could be an issue?

I just prefer the xfce UI.

[maxnet]

I got the same message about sockets.

Note that it are warnings that those files mentioned will not make it into the .tar
Does not affect the creation of the rest of the image though.

So if it concerns software that is not essential for your needs, you can just ignore the warnings.

[MilesAheadToo]

Yeah, well the RPi does not start, but if I use one of the standard RaspOS images on the same RPi, it does start.

[maxnet]

I start from the RaspOS Lite and then run the following:

sudo apt update -y
sudo apt upgrade -y
sudo apt install -y xserver-xorg xfce4 xfce4-goodies xrdp gparted
sudo dpkg-reconfigure lightdm

If this is all you want to do, do not bother with custom images.

• Install normal RPI OS lite
• In the software tab, select the OS you just installed and press the "shell" button.
• Do your customizations there from shell.
• Everything you execute runs as root. So leave out the "sudo"

[MilesAheadToo]

That is very cool!! Thank you.

[MilesAheadToo]

This works great, expect I get the error when I try to set the display manager to lightdm using the command dpkg-reconfigure lightdm. The image does start, but never completes. I just see a blinking cursor in the top left of the screen.

I'm following these instructions: Install XFCE desktop on your Raspberry Pi

dpkg-reconfigure lightdm
Running in chroot, ignoring request.
[....] Reloading system message bus config...Failed to open connection to "system" message bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
invoke-rc.d: initscript dbus, action "reload" failed.

[maxnet]

For lightdm you probably need to execute the following 2 commands that are normally done by convert script:

echo "tmpfs           /var/lib/lightdm  tmpfs defaults,mode=755,nosuid     0       0" >> /etc/fstab
sed -i "s/#user-authority-in-system-dir=false/user-authority-in-system-dir=true/" /etc/lightdm/lightdm.conf

Not sure if anything extra is needed for XFCE.
Haven't tried.

[MilesAheadToo]

Ugh. I waited too long before I cloned the starting image, namely Raspberry Pi OS Lite, to something else before making changes. Of course I should have cloned it immediately, and then made changes to the clone. But I didn't.

And now I get an error when I try to delete the OS from PiServer. The Clients are pointed to the Raspberry Pi OS image, so that isn't the issue.

And I can't overwrite the OS by loading the original with the same name.

And suggestions other than removing PiServer and reinstalling it?

[maxnet]

And suggestions other than removing PiServer and reinstalling it?

1. reboot first, to make sure nothing is still using that folder
2. if it still not works, try deleting the folder manually from a normal shell (not inside piserver). rm -rf /var/lib/piserver/os/name-of-os

[MilesAheadToo]

And suggestions other than removing PiServer and reinstalling it?

1. reboot first, to make sure nothing is still using that folder
2. if it still not works, try deleting the folder manually from a normal shell (not inside piserver). rm -rf /var/lib/piserver/os/name-of-os

I feel like an idiot: I still had the shell for the Raspberry Pi OS Lite open while trying to delete it. Closing the sheel solved the issue.

[MilesAheadToo]

Update: I have been able to install XFCE successfully. Thank you very much for your help.

Question: How do I enable SSH?

However: I am struggling to get RDP working.

I have created these images:

XFCE Desktop - Base is named incorrectly. It is really Raspberry Pi OS Lite on which I have run update and upgrade.
For XFCE Desktop I cloned XFCE Desktop - Base and installed xserver-xorg, xfce4, and xfce4-goodies. I used your instructions to set lightdm. It works perfectly from a screen attached to the RPi.
For XFCE RDP I cloned XFCE Desktop and installed xrdp. I couldn't login.

Just to be sure I wasn't doing anything stupid, I cloned Raspberry Pi OS to RPiOS RDP and installed xrdp. I couldn't login.

[maxnet]

SSH has the issue you need to create an unique host key first.
In the chroot shell in Piserver GUI:

ssh-keygen -A
systemctl enable ssh

XRDP has the issue that it wants to write a log file on the read-only file system, and does not like that is not going to work.
To counter tell it to log to /dev/null:

apt install xrdp
sed -i "s#LogFile=xrdp.log#LogFile=/dev/null#" /etc/xrdp/xrdp.ini

(For any other issues, please create a new issue or ask on forum.
Because you are replying in somebody else's existing issue, the original author of the issue, will get an e-mail notification about every reply. And not sure if he/she is interesed in things like xrdp)