Fedora Downstreams Made Easier

About six years ago, I quit my dream job at Red Hat to physically move to Germany. An unexpected casualty of that move was that my contributions to Fedora dried up. I kept running Fedora on all my systems, and also kept in touch with the Indian and APAC ambassadors; even attended part of the Flock in Dresden – but working closely with the project suffered as I navigated the uncharted waters of a new company and a new country.

I joined AWS to work on the kernel and hypervisor components of all internal Amazon computers. Sooner or later, that would mean I would end up working on Amazon Linux, the distribution created by former Fedora Project Leaders within Amazon. That opportunity came recently, when I got involved with shaping the soon-to-be-released Amazon Linux 2022.

While working on defining package sets, which binaries to include in the default set, which compiler options to pick, and such, I noticed we were embedding logic in %{?amzn} conditionals in the spec files; and we were also replicating many of the settings already used in %{fedora >= ...} or %{?rhel} conditionals.

I also noticed very awkward spec files, like the following snippet from clang.spec:

-DCLANG_REPOSITORY_STRING="%{?fedora:Fedora}%{?rhel:Red Hat} %{version}-%{release}" \

or the following from gdb.spec:

# Change the version that gets printed at GDB startup, so it is RH specific.
cat > gdb/version.in << _FOO
%if 0%{!?rhel:1}
Fedora %{version}-%{release}
%else # !0%{!?rhel:1}
Red Hat Enterprise Linux %{version}-%{release}
%endif # !0%{!?rhel:1}
_FOO

To add Amazon Linux-specific strings here would have made the spec files fun to read and maintain (not!). There was similar code also in gcc.spec and elfutils.spec.

I thought it’d be better - not just for Amazon Linux - but for Fedora’s spec files in general - for us to make the spec files easier to read. These oft-repeated constructs could be put away in distribution-specific macros. CentOS could benefit from it as well - where CentOS-specific customizations further uglify the spec files.

To fix this situation, I came up with two main approaches, both with their pros and cons:

1. A new set of macros get defined based on which distribution we’re compiling this package for. Values are picked up from system macros. As an example, a new distro_vendor macro, available at build time, will expand to Fedora on Fedora systems; Red Hat on RHEL systems, and ‘Amazon Linux` on AL systems.

   The implementation of this set of macros looks like:

    %distro_vendor %{lua:
    local vendor = 'Fedora'
    if macros.rhel then
        vendor = 'Red Hat'
    end
    if macros.amzn then
        vendor = 'Amazon Linux'
    end
    return vendor
    }
   

   If CentOS wanted to add its own customizations, it would add another if case here, and all users of the distro_vendor macro would then get updated. The previous example, from clang.spec, will change to this simple string now with such a macro:

   -DCLANG_REPOSITORY_STRING="%{?distro_vendor} %{version}-%{release}" \
   

   and this will work out of the box on Fedora, RHEL and AL.

   Pro:

   • All customizations contained in one file - no further downstream customizations necessary.

   Con:

   • Fedora becomes the upstream for downstream-distribution-specific macro definitions

2. A new distribution-specific file where the distribution defines its macros as it pleases. Fedora does not need to worry about its downstreams’ definitions; and downstreams in turn can just patch this one file to define its own distribution-specific values:

   %global dist_vendor Fedora
   

   That’s it - that makes the dist_vendor macro available to all spec files.

   Pro:

   • Fedora defines distribution-specific macros in one place; does not worry about downstream definitions
   • Much simpler implementation than lua scripts for RPM macros

   Con:

   • Each downstream distribution initially needs to define some key macros right away; currently properly-building binaries will produce undesirable output (eg failed builds) if these macros aren’t defined.

   Due to the big Con here, this was my 2nd approach, even though it was technically much better.

I posted both these approaches as PRs: first and second.

The first approach was disliked by all – the second one won. However, implementing this was challenging, as we needed to update Fedora, CentOS, and RHEL at the same time to ensure spec files and their backports kept working properly.

Thankfully, Fedora and CentOS engineers helped me push through the changes there; and Stephen Gallagher additionally wrote Red Hat deinitions for RHEL and pushed them through there.

In a surprisngly short amount of time, Fedora (F35+), CentOS 9 Streams, and RHEL have these macros available, as does Amazon Linux 2022. Package maintainers can use these macros in Fedora, and expect them to work right for any downstream distributions, without having to submit modified PRs for those.

To get a sense of how simple things become with such a change, the gdb.spec file mentioned above gets simplified to:

%{?dist_name} %{version}-%{release}

Another favourite example of mine is this change to the elfutils.spec:

-%configure CFLAGS="$RPM_OPT_FLAGS" --enable-debuginfod-urls=https://debuginfod.fedoraproject.org/
+# dist_debuginfod_url is defined in macros.dist. Fedora and CentOS have
+# URLs pointing to their respective servers.  RHEL and Amazon Linux do
+# not configure a default server.
+%if "%{?dist_debuginfod_url}"
+%configure CFLAGS="$RPM_OPT_FLAGS" --enable-debuginfod-urls=%{dist_debuginfod_url}
+%else
+%configure CFLAGS="$RPM_OPT_FLAGS"
+%endif

This enables all these distributions to now share the spec file, and have them pick up the right default for what makes sense for that distribution.

The complete list of macros defined, with the Fedora defaults is:

While the right technical solution won at the end, I’m happy I got to explore RPM macros and Lua scripting – I’d never attempted that earlier, and figuring that out was quite a fun exercise!