Fixed firmware cleanup script (#1812)

## Problem

- See #1756
- Some firmware files were missing in the Live ISO

## Solution

There were basically two problems:

1. Few kernel modules use wildcards to specify the firmware files, e.g.
`ath11k/WCN6855/hw2.1/*`. That was not supported by the script, it
expected only individual file names referenced by drivers.
2. Some firmware files use symlinks, e.g. the driver refers to `foo.bin`
file but in some cases it is actually a symlink pointing to
`foo-1.0.bin`. The script then deleted the `foo-1.0.bin` file as it
looked like not used by drivers. The result was a dangling symlink and a
missing firmware file.

Both issues were fixed. The glob patterns are evaluated using the
`Dir.glob` function and if the referenced file is a symlink then it also
keeps the target file where the symlink points to.

## Testing

- Tested manually, the firmware for Atheros cards is now present
- I compared the result with installation-images and now we basically
include the same firmware as installation-images do in the classic
Tumbleweed installation
- The ISO size increased just by ~13MB, fortunately not that much...

---------

Co-authored-by: Imobach González Sosa <igonzalezsosa@suse.com>

live/root/tmp/fw_cleanup.rb

@@ -8,30 +8,82 @@
 require "find"
 require "shellwords"
 
+# really delete or just do a smoke test?
+do_delete = ARGV[0] == "--delete"
+
+# firmware location
 fw_dir = "/lib/firmware/"
 
+# in the Live ISO there should be just one kernel installed
 dir = Dir["/lib/modules/*"].first
 puts "Scanning kernel modules in #{dir}..."
 
-# list of referenced firmware names from the kernel modules
+# find all symlinks until the final file is found, if a file is passed then the
+# file only is returned
+# @return [Array<String>] list of files/symlinks
+def symlinks(path, fw_dir)
+  ret = [path]
+
+  while File.symlink?(path)
+    # do not use File.realpath as it skips all intermediate symlinks and returns the final target
+    target = File.readlink(path)
+    abs_target = File.absolute_path(target, File.dirname(path))
+
+    # a cycle or a broken link is detected, or the link points outside of the firmware directory
+    break if ret.include?(abs_target) || !File.exist?(abs_target) || !abs_target.start_with?(fw_dir)
+
+    puts "Adding symlink #{path.delete_prefix(fw_dir)} -> #{target} (#{abs_target})"
+    ret << abs_target
+
+    # prepare for the next iteration round
+    path = abs_target
+  end
+
+  ret
+end
+
+# list of referenced firmware names from the kernel modules, includes symlinks and expanded globs,
+# the paths are relative to the /lib/firmware root
 fw = []
 
+# traverse the kernel drivers tree and extract the needed firmware files
 Find.find(dir) do |path|
   if File.file?(path) && path.end_with?(".ko", ".ko.xz", ".ko.zst")
-    fw += `/usr/sbin/modinfo -F firmware #{path.shellescape}`.split("\n")
+    driver_fw = `/usr/sbin/modinfo -F firmware #{path.shellescape}`.split("\n")
+
+    driver_fw.each do |fw_name|
+      f = fw_name.dup
+      # the firmware can be compressed, match also the compressed variants, if
+      # a wildcard is at the end then it already matches so it does not need
+      # to be added (compressed files would be then included twice)
+      f += "{,.xz,.zst}" if !f.end_with?("*")
+
+      matching = Dir.glob(File.join(fw_dir, f))
+
+      if fw_name.include?("*")
+        puts "Pattern #{fw_name.inspect} expanded to: #{matching.join(", ")}"
+      end
+
+      # find symlink targets if the file is a symlink
+      fw_files = matching.inject([]){|acc, file| acc += symlinks(file, fw_dir)}
+
+      fw += fw_files.map{|m| m.delete_prefix(fw_dir)}
+    end
   end
 end
 
+# counter for total unused firmware size
 unused_size = 0
 
+# traverse the firmware tree and delete the files which are not referenced by any kernel module
 Find.find(fw_dir) do |fw_path|
   next unless File.file?(fw_path)
 
-  fw_name = fw_path.delete_prefix(fw_dir).delete_suffix(".xz").delete_suffix(".zstd")
+  fw_name = fw_path.delete_prefix(fw_dir)
 
   if !fw.include?(fw_name)
     unused_size += File.size(fw_path)
-    if (ARGV[0] == "--delete")
+    if (do_delete)
       puts "Deleting firmware file #{fw_path}"
       File.delete(fw_path)
     else
@@ -40,4 +92,12 @@
   end
 end
 
-puts "Unused firmware size: #{unused_size} (#{unused_size/1024/1024} MiB)"
+# do some cleanup at the end
+if (do_delete)
+  puts "Removing dangling symlinks..."
+  system("find #{fw_dir.shellescape} -xtype l -print -delete")
+  puts "Removing empty directories..."
+  system("find #{fw_dir.shellescape} -type d -empty -print -delete")
+end
+
+puts "Unused firmware size: #{unused_size} (#{unused_size >> 20} MiB)"

live/src/agama-installer.changes

@@ -1,3 +1,10 @@
+-------------------------------------------------------------------
+Thu Dec  5 11:19:10 UTC 2024 - Ladislav Slezák <lslezak@suse.com>
+
+- Fixed missing firmware for some drivers (gh#agama-project/agama#1756)
+  - Support wildcards ("*") in the references firmware files
+  - Handle symbolic links
+
 -------------------------------------------------------------------
 Tue Dec  3 12:41:31 UTC 2024 - Imobach Gonzalez Sosa <igonzalezsosa@suse.com>