paper.tex

\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}
\documentclass[a4paper]{article}
\usepackage{authblk}
\usepackage[alldates=iso,seconds=true,sorting=none]{biblatex}
\usepackage{graphicx}
\usepackage[margin=2.5cm]{geometry}

\DeclareUnicodeCharacter{200B}{{\hskip 0pt}}

\usepackage{libertine}
\usepackage{orcidlink}
\renewcommand\Affilfont{\small}

\usepackage{hyperref}
\urlstyle{same}
\usepackage{xspace}
\usepackage{caption,tikzpagenodes,everypage,ifthen}

\newcommand*{\eg}{e.\,g.\@\xspace}
\newcommand*{\ie}{i.\,e.\@\xspace}

\hypersetup{%
  %pdftitle={},
  %pdfkeywords={},
  colorlinks=true,
  linkcolor={darkgray},
  filecolor={Maroon},
  citecolor={Blue},
  urlcolor={Blue},
}

\usepackage{todonotes}

\addbibresource{bibliography.bib}
\addbibresource{positionpaper.bib}
\title{Establishing central RSE units in German research institutions}

\include{contributors.tex}

\date{2024--12--31}

\include{marginplots.tex}

\begin{document}

\maketitle

\section{Introduction}
Research software has been written and used for decades in a increasing range of disciplines.
It has been established that most research requires research software for its results~\autocite{Hannay2009, Hettrick2015}.
To solve pressing research challenges, better software is crucial~\autocite{Goble2014}.
During the past decade, it gained ever-growing attention and is becoming accepted as a research result on its own.
%We follow here the definition: “Research Software includes source code files, algorithms, scripts, computational workflows and executables that were created during the research process or for a research purpose”, with full definition and discussion provided in~\autocite{Gruenpeter2021}.

The number of people developing software in academia is constantly rising~\autocite{Hannay2009, Hettrick2015}.
Research Software Engineering encompasses creating, adapting or maintaining Research Software.
It also includes consultation and training with the aim to enable researchers to some of these tasks themselves.
These actions are very diverse and so are the environments they are performed in.
This position paper focuses on groups of research software engineers that provide their services for an entire research organisation or at least a substantial part of it.

We advocate the establishment and support of dedicated, central RSE groups in German research organisations, with clearly defined tasks, contact points, and, in particular, sustained funding, for the benefit of all researchers in their organisation.
We provide an overview of the various tasks these teams have and discuss potential realisation strategies, learning from already existing RSE units.

Depending on the national research environments and processes that readers are familiar with, the notion of the terms \emph{software} and \emph{research} might differ.
The term “research software” has also no single definition within the community.
Therefore, to avoid ambiguities, we list the definitions hat we use in this document:\\
\textbf{Software:}\\
Source code, documentation, tests, executables, and all other artefacts that are created during the development process and that are necessary to understand its purpose.\\
\textbf{Research software:}\\
Foundational algorithms, the software itself, as well as scripts and computational workflows that were created
during the research process or for a research purpose, across all domains of research.
This definition is broader than in~\autocite{FAIR4RS} and is the outcome of a recent discussion in~\autocite{Gruenpeter2021}.\\
\textbf{Research software engineers:}\\
People who create or improve research software and/or the structures that the software interacts with the computational ecosystem of a research domain.
They are highly skilled team members who may also conduct their own research as part of their role.
However, we also recognise RSEs who have chosen to focus on a technical role as an alternative to a traditional research role.\\
\textbf{Researchers:}\\
RSEs might also be researchers.
However, for the lack of a proper term and to avoid many “non-RSE researchers” within the text, we will refer by “researchers” to all non-RSEs involved in research or in research supporting organisations such as in \eg{} libraries, hence those that are at most sporadically performing RSE actions.\\
\textbf{RSE Hub}:\\
This is our general term for the central RSE team throughout this paper.
These RSE Hubs can take the form of, \eg{} full RSE units, smaller RSE groups, Open Source Program Offices (OSPOs), virtually across multiple units or combined under single leadership,
 depending on the environment of the hosting research organisation.
All of these implementations are considered, taking into account the large variety of research environments in Germany which include not only universities, but also research consortia and other research performing organisations.

\section{Motivation for central RSE units}
\begin{quotation}
      \textit{Better Software, Better Research}\\(Mission statement of the UK Software Sustainability Institute)
\end{quotation}

The quote above is the shortest possible summary of this chapter: most if not all motivation to provide RSE services stems from the goal of improving research.
Tasks RSEs perform include training, \eg{} to improve the often low-quality code developed by beginners~\autocite{Ostlund2023}, consultation services, \eg{} regarding frameworks or algorithm selection, and the development of existing or new software.
For an overview of typical tasks of RSEs and the competencies required, see~\autocite{goth_foundational_competencies_2024}, especially section\ 4.4:\ “RSE tasks and responsibilities”.

\subsection{Pooling: a necessary ingredient}
The main focus of this paper lies on central RSE teams since the benefits of RSEs to research are described elsewhere **TODO: [REF]**.
The main advantages of central RSE units all stem from the pooling of resources.
There are at least three aspects to RSE pooling that research institutions can benefit from: funding, diverse knowledge, and support contacts.
The first, pooling of \textbf{funding}, allows organisations to invest in building up institutional knowledge by supporting RSEs to become experts.
A central RSE team on long-term contracts will act as a knowledge hub due to their accumulated experience in and support of several disciplines as well as established contacts within the organisation.
This is comparable to commercial/industry R\&D departments, where key software architects and developers establish a knowledge hub and consult with as many projects as necessary \todo{Does that exist in reality? Isn’t it just that people, on average, stay longer?} [REF].
% side-note: it's also similar to “inhouse consulting” in management\autocite{moscho_inhouse_consulting_2010}. They even formed a national network to raise awareness about the internal consultant role (https://inhouse-consulting.de/).
Subject matter experts like software architects, database administrators and other tooling specialists are organised centrally and share their knowledge with members of decentralised projects.
It makes economic sense to organise such staff centrally since not every project has a need for a full-time specialist or can afford one over an extended period of time.
Most academic research organisations have established centralised tooling, \eg{} storage or High-Performance-Computing\ (HPC), but only a few consider software development and consultancy a relevant service yet.

A second and equally important aspect to pooling RSEs is that of \textbf{diverse knowledge}.
RSE units act as knowledge hubs in a network of academic developers within an organisation~\autocite{Elsholz2006}.
Groups of RSEs with tasks spanning the entire organisation necessarily have to offer diverse knowledge.
Obtaining this diversity of knowledge and experience can also be a challenge, but once it has been established it quickly becomes an asset to the organisation.
RSEs in centralised groups are interdisciplinary specialists due to their experience working on diverse topics, \todo{FLO+PMS: should be reformulated, guess: RSEs know diverse research methodologies + general RSE knowledge} as well as overlaps in methodology across disciplines and research software in general.
They are assumed to be able to suggest the most appropriate tools/frameworks and design or architecture patterns for certain research challenges.
Their diversity in skills (languages, frameworks, front/back-end, UX, management) is welcomed, especially for short-term needs in projects.
This will improve the chances of a project being successfully completed in a timely manner.
That means a central RSE unit has more RSE competencies than any individual research group in the institution.
This allows members of that unit to bring in new ideas or transfer them from other collaborations to these groups.

The third aspect to pooling RSEs is visible most of all from a users perspective: a \textbf{single, central contact point} for digital challenges is valuable to researchers, whose first problem often is not knowing whom to contact, partially because while they know what they want, they might not know what they need.
A central RSE team can, due to its proximity to research, much better listen to the wishes expressed by researchers and then help formulate needs and act as a channel to either fulfil them themselves or reformulate and redirect the request.
The results are increased research speed and quality and with that a higher reputation of the entire research organisation.

%In this chapter, we motivate dedicated RSE groups in German research organisations.
%Several stakeholder perspectives are discussed and supported by (inter)national examples, including that of RSEs within RSE groups, RSEs embedded in research groups, Researchers in need of RSE resources, organisational management and that of funders.
\subsection{Pooling: an already tested idea}
The idea to pool resources in specific areas within an organisation is not new.
For example, similar arguments can and have been made for research data support.

\subsubsection{Research data management}
Both data and software play a fundamental role in almost all of research.
Over the past decades, Research Data Management (RDM) has evolved into a topic of national interest with NFDI consortia for all disciplines and a research data law.
Federal state RDM initiatives\footnote{\url{https://forschungsdaten.info/fdm-im-deutschsprachigen-raum/deutschland/}}\todo{PMS: Make footnotes proper citations?} have established the topic further and provide regional training, networking and other supporting services.
Many research organisations have set up established central RDM groups that support research projects in all aspects from grant proposals to hands-on support and maintaining Data Management Plans (DMPs).
Funding agencies acknowledge the importance of research data and have started to make RDM mandatory in research projects.

The most recent funding guidelines suggest to have “data stewards” in data-driven research.
Such experts are to be employed in advanced research projects like “Collaborative Research Centres” (CRC)\footnote{Sonderforschungsbereich (SFB)} or “Clusters of Excellence”\footnote{Cluster der Exzellenzinitiative}.
These data experts support research projects in several aspects including DMPs, grant applications, data availability for journal publications, compliance, FAIRification and more.
Similarly, central RSEs will encourage other RSEs to publish software with rich metadata and will support journal publications with code submission requirements.
With the increasing recognition of software as a research product, it is easy to see how projects will require and benefit from support in research software management in the near future.

Due to the similar nature of both, data and software, and their importance in today's digital research, it is reasonable to expect a similar trajectory in the development of research software as a topic, as we have witnessed for research data.
%such output has become more important over the last two decades. Since research software can be considered valuable research output as well, we expect a similar trajectory for software.
%gh training researchers, the reusability through data repositories and to avoid duplication of effort.
%For over a decade, research funders and organisations made a significant effort to establish RDM and teams around it, for example the Utrecht University Research Data Management Support~\autocite{UtrechtRDM}, University of Stuttgart FoKUS team~\autocite{Boehlke2024} or TUBS.researchdata~\autocite{Grunwald2022} at TU Braunschweig.
%We assume that research software will follow a similar trajectory.
%\footnote{For arguments when research software is unlike data, see \autocite{Lamprecht2020}.}

\subsubsection{Existing RSE efforts}
Some organisations already have central RSE teams.
Examples \todo{FLO+PMS: Do we want to mention all? DLR has something similar. So far it’s only the authors’ group.} of organisational RSE teams in Germany are
 the Helmholtz HIFIS group\footnote{\url{https://events.hifis.net/category/4/}}\autocite{haupt_hifis_consulting_2021},
 the Scientific Software Center in Heidelberg\footnote{\url{https://www.ssc.uni-heidelberg.de/en}}\autocite{ulusoy_heidelberg_ssc_2024},
 the Competence Center Digital Research (zedif) in Jena\footnote{\url{https://www.zedif.uni-jena.de/en/}},
 and the SURESOFT workshops series in Braunschweig\footnote{\url{https://suresoft.dev/}}~\autocite{Blech2022}.
Another national pioneer is the Göttingen State and University Library.
%which set up a group of RSEs offering – besides training – services like data modeling and visualization, digital editions, portal development and more.
The latter reported a remarkable increase in software quality, better grant applications, less brain drain and overall better employee satisfaction levels~\autocite{schimavoigt2023}.
%The demand for such services appears to be ever-increasing.
%Other tasks include code review (REF? Charite), consultation services regarding frameworks or algorithm selection, licensing, and more.
%RSEs have always embraced and supported collaborative infrastructure and tools, \eg{} GitLab, Containerisation, etc.\ and thus enabled fellow researchers utilising such infrastructure.
In some national and international organisations, established RSE groups already develop solutions for (and guided by) research projects.
%This approach assures high quality research software and allows domain scientists to focus on their research challenges.
%This is likely to save time and accelerate publication of results.

While we focus on Germany here, it is beneficial to review how other countries approach research software.

%\subsection{Structure}
%Furthermore, given appropriate long-term funding, a central RSE hub will be able to keep essential software alive, even if it was developed in short-term projects.
%Such code often requires long-term maintenance, support, new features or bug fixes.
%The decision of curation is commonly based on measures that involve quality, academic or societal impact among many others.

\todo{FLO+PMS: They are not an example for an institutional RSE structure. We can scrap this, paragraph starting here} The Carpentries\footnote{\url{https://carpentries.org}}\autocite{Wilson2006} exemplify a similar success story\footnote{Carpentries25 Testimonial Series: \url{https://carpentries.org/blog/tag/carpentries25/}}.
\todo{Assertion-citation mismatch.} Requests or suggestions for even more training show the need for such services\footnote{Carpentries Incubator and Carpentries Lab: \url{https://carpentries.org/lesson-development/community-lessons/}}.
RSE services which benefit all disciplines/departments may represent a unique selling point for organisations competing for the brightest minds, see the examples from leading German research performing organisations above.\todo{PMS: Scrap whole paragraph except for first sentence?}

%Given that RDM training or coordination is a centralized effort in most organisations, the time has come to implement a similar structure for research software services.
%Such a group may extend or include RDM or collaborate with such service teams.
%See the Vision and Realization sections below for more details.

In the UK, for example, many universities started initiating dedicated RSE units about a decade ago~\autocite{Crouch2013}.
The successful establishment of these units is a role model for similar research performing organisations worldwide.
A range of already-existing RSE units can be seen in this map: \url{https://society-rse.org/community/rse-groups/} \todo{FLO+PMS: mention that this map is not current and add further data.}.

In the UK, for example, almost all grant applications include software development in their budget.
This allocated money can then be utilised to delegate/dispatch a central RSE person or group into a research project for a few weeks or months as necessary.

The HPC community,
arguably a sub-community of the RSE-community,
has a history of offering training and consulting to prospective and active users of the hardware they support.
National Competence Centres\footnote{EuroCC: \url{https://www.eurocc-access.eu/}} form a network of HPC-RSE consulting groups to share expertise with academic and industry actors\autocite{eurocc_success_stories_2023,eurocc_success_stories_2024}.

\subsection{External expectations}

The latest DFG grant application templates require discussion of both, data \textbf{and} software management (in line with their GWP guidelines~\autocite{dfg_gsp}).
%We also see the first grant applications [REF wellcome trust (seems to require an OutputMP that includes software, but only how/when it will be published, not how it will be created/maintained)? or others] requiring Software Management Plans (SMP).
In addition, as dedicated DMPs have become mandatory in several funding calls \todo{REF}, we expect to see a similar development for Software Management Plans (SMPs) in the future. \todo{REF} There have already been funding calls in the UK that required an SMP.
% See https://www.forschungsdaten.org/index.php/Data_Management_Pl%C3%A4ne#Anforderungen_von_F%C3%B6rderorganisationen
% See https://www.researchdata.uni-jena.de/information/datenmanagementplan

Policies for research software management and guidelines involving responsible research practices detailing software handling are the precursors for a research software engineering environment.
See for example position papers by the Helmholtz Open Science Office~\autocite{Helmholtz2019a,Helmholtz2019b},
the AllianzInitiative~\autocite{Konrad2021},
the University Utrecht~\autocite{Utrecht2016b},
and the German Research Council~\autocite{dfg_gsp}.
%% TODO: Double-check that DLR guidelines are referenced.

Another development taking place worldwide is the encouragement of authors to submit both, data and software, for peer review.
As an example, the journal “Nature” initiated such a policy\footnote{\url{https://www.nature.com/nature-portfolio/editorial-policies/reporting-standards}} in 2018~\autocite{Nature2018}.
RSE groups are able to offer researchers consulting tailored to their specific needs on how to implement and document those policies.

The global FAIR movement originated from RDM and widened their focus to include research software.
However, it also has become clear in that process that software is not “just another type of data” and that the FAIR principles are not sufficient for software.
The FAIR principles for Research Software (FAIR4RS)~\autocite{ChueHong2022} have been adopted worldwide~\autocite{Barker2024}, including the German Ministry of Education and Research (BMBF) and the German Research Foundation (DFG).
% adoption of FAIR4RS (inter)nationally
The rather complex assessment of FAIRness~\autocite{Wilkinson2023,FAIRmaturity} has also widened from data to software~\autocite{Lamprecht2020}.

%\subsection{Better Software, Better Research}
%Such software is assumed to have a much longer life cycle and may be more evolvable or extensible due to better code quality and architectural decisions that ease reuse.

%\subsection{Towards a Thriving Future}
%
%In conclusion, we observed that RSE groups already do support research software development, publication, and development among many important tasks.

%Publication efforts for better software will increase discoverability which in turn will decrease duplication of effort.
%Scarce resources like professional staff, time and money are not put to waste.
%Instead, better software (publications) will lead to outstanding reputation.

%A professionalization in software development and management can be expected to improve the transition from prototypes to software products.% to the benefit of everyone.
%Less technical debt\footnote{\url{https://www.gartner.com/en/information-technology/glossary/technical-debt}} will be amassed, which is beneficial for reuse.

%High-quality software is likely to be published, cited, and reused.
%Better software is assumed to have a much longer life cycle and may be more evolvable or extensible due to better code quality and architectural decisions that ease reuse.
%This will benefit researchers, organisations, funders, and society.

%Necessary software management activities like (git-based) version control are assumed to improve collaboration among researchers.




% move to vision
%Software development and management training efforts included in the "studium generale" (or similar education strategies) will further the knowledge of students and early career researchers.

%In contrast, organisations lacking such RSE knowledge need to purchase professional support - in the industry often realized via consulting.
%Academic research hardly has the resources to compete for effective consulting.
%Academic research is assumed to aim for sovereignty and independence of third-party providers.

\section{Vision}%
\label{sec:vision}

In this section, we describe our vision for central RSE units at research institutions in Germany.
Institutions is, for our purposes, a broad term including universities, other colleges, associations like Max-Planck, Helmholtz, Fraunhofer, or Leibniz, as well as other research performing organisations.
They show a wide variety in organisational structure as well as internal scientific diversity.
Thus, there can be no single optimal blueprint for such an RSE unit for all research institutions in Germany.
We instead describe modular components that can be mixed and matched based on the respective local environment.

We have identified nine modules that can make up an RSE unit.
In practice most RSE units will only support some of these modules.
Different RSE units will focus on different sets of modules.
Thus, it is likely that no two RSE units will be, or should be, alike.
However, these nine modules together with assumed weights are part of a simple model of an RSE group which provides both a quick overview of an individual group as well as a way to compare groups.
The nine modules are described below.
\todo{FL:\@ Introduce the hub-spokes terminology.}

\subsection{Module 1: Foster a local Network of RSEs}%
\label{sec:network}

One of the core responsibilities of an RSE unit is to act as a coordinator of RSE activities within the institution.
At virtually every academic institution there are employees that assume at least part-time the role of an RSE.
These RSEs typically work isolated from similar RSEs in different groups, within the same institution.
A central RSE unit provides a condensation core and connects RSEs in spokes with each other and with those at the hub.

Connecting RSEs in spokes has multiple, positive effects, both for them as well as for the organisation.
It will enable them to get to know others in similar situations and to learn from as well as support each other.
Contact with the central RSE unit will also help RSEs to professionalise their software-related work, which will directly benefit not only themselves but also their research groups.
In addition, the networking opportunities allow the distribution of knowledge about tools and resources within network partners, including the central RSE unit.
There are many RSE skills for which mastering can take many years; time that a part-time RSE usually can not spare.
A central RSE unit can make sure to connect RSEs in spokes to others with the relevant expertise or offer it themselves.
Fostering the network also enables the RSE unit to monitor institutional RSE activities, thereby providing the insight necessary to prevent duplication of work and support synergies.

How an RSE unit realises this task will depend heavily on its environment and resources.
We only mention a few examples here to provide inspiration, with the explicit claim of incompleteness.
These include talks, seminars, workshops, meet-ups, hackathons, as well as informal regulars' tables.
As a foundation, a central RSE unit employs experienced RSEs, mostly at the post-doctoral level, who are not only expert software engineers, but also good communicators with the ability to work interdisciplinarily.
At least a core of a central RSE unit's employees need to have permanent contracts to be able to offer that deep expertise that requires years of experience.
Moreover, an onboarding process can serve as an entry point for new RSEs, whether in the hub or in spokes, into an institution's network.
This gives an opportunity to gauge how the new colleague can benefit from the RSE unit's teaching services and whom they might want to network with based on their planned work.
Similarly an off-boarding process can help to make sure that all acquired knowledge that is relevant to the institution is passed on to someone who stays, even if possibly not within the specific research group.

\subsection{Module 2: Consultation Services}%
\label{sec:consultation}

With the majority of researchers being self-taught programmers~\autocite{Carver2013}, there is a huge demand for expertise on how to develop better research software.
Here, “better” can refer to a number of quality metrics such as correctness, reproducibility, maintainability, extendability, usability, portability, interoperability, performance or scalability~\autocite[Chapter 16]{Schulmeyer2008}.

In order to raise the quality standards for research that is based on research software, it is of great importance for research institutions to provide access to such expertise with a low barrier to entry.
The hub is a natural place to provide this central service.
There exists a number of scenarios where RSE consultation services differ strongly in scale and format.
We mention a few of these in the following.
“One Off” consultations on any research software related aspect that are open to researchers of all career levels are
a great introduction to the hub's RSE services and are offered by almost all RSE units already established [REF].
%Depending on the demand, these consultations can either be by appointment or in a more structured format where you book an appointment from available dates (\eg{} University of Sheffield's “Code Clinic”\footnote{At time of publication the appointment form could be access from the front page of the RSE unit’s website: \url{https://rse.shef.ac.uk/}} and Friedrich Schiller University’s Digital Research Clinic\footnote{At the time of publication upcoming clinic’s were advertised on the consulting page of the Competence Center For Digital Research’s website: \url{https://www.zedif.uni-jena.de/en/consulting.html}}).

A larger scale format for RSE consultation services could be that a research project regularly meets with an RSE in order to coordinate the research software efforts done in the research project.
This format enables valuable feedback cycles between researchers and RSEs and allows RSEs to guide the project
towards successful software engineering best practices without overloading the researchers with information at a one-off consultation.
When an RSE unit carries out many of these project consultations, they will gather valuable experiences in transferring RSE knowledge to practitioners.
An RSE hub puts these experiences into institutional memory, allowing for better RSE practice in the future.

RSE consultation services are also of great importance in proposal writing.
Many proposals critically depend on research software to be developed and the requirements of funding agencies with respect to research software are growing and will continue to do so.
Similar to dedicated RDM units that provide institutional support for data management plans,
the RSE hub can support researchers by providing expertise with software management plans and the software engineering best practices required by these plans.
Consultation services that are already involved in the proposal phase are expected to lead to an improved research workflow and thereby to better research.
This should lead to improved proposal acceptance rates, thereby amortising the initial investment into RSE units.


\subsection{Module 3: Development Services}%
\label{sec:development}

There is a huge demand for the development and customisation of research software tailored to the needs of specific research projects.
In many cases this can be achieved by educating researchers to write their own software according to RSE best practices.
Bringing in expert RSEs to deal with the development of complex research software frees up the researchers to focus on research.
In these cases, RSE development services are a great opportunity that can have a huge impact on the digitisation of science at a research institution.
Similar to consultation services, this service can be offered at multiple scales.

Many times, even a small effort of a skilled RSE can have a huge impact on a research project that requires dedicated research software development.
With the leverage of these projects being usually very high, realising as many of them as possible gives a great boost to the research institution.
Many existing RSE units (\eg{} Manchester, Heidelberg) offer this type of small scale service free of charge and use it to promote their services within the institution.

For research projects requiring more substantial software development resources, an RSE unit could --- either through the hub or its spokes --- provide the required developer capacity.
This is especially relevant if the researchers hired for the research projects do not have the required software development skills and the volume of the development is too small to hire a dedicated developer.
Depending on the scale of the involvement, the RSE unit can either be included into the grant proposal via a co-PI or as an internal service provider.

If the research within an institution heavily relies on specific pieces of software,
it is of vital importance for the long term success of the institution to sustain these pieces of software.
Relying on a workforce that is subject to academic labour turnover poses a risk of knowledge loss.
If the development is done in an RSE unit with long-term contracts, institutional memory about critical research software infrastructures can be created and the long term availability of these infrastructures can be improved.
This applies both to domain-specific research software (\eg{} simulation frameworks widely used throughout the institution)
and to domain-agnostic software and data infrastructure (\eg{} Jupyter, workflow management systems, data repository software).

While all of the above development services can be flexibly performed either at the RSE hub or its spokes, there are advantages of having a hub in the process:
It allows building up highly specialised technical expertise with a long term perspective and reuse it across the entire institution.
Examples of topics that would benefit from such expertise pooling are \eg{} mobile app development and UI/UX development.

RSE units that offer development services at all scales have proven to be a success story at many research institutions and have rapidly grown in size due to the influx of third party funding.
Notable examples are \eg{} Manchester~\autocite{Sinclair2022}, Notre-Dame [REF], Stanford~\autocite{Stanford2023}, Princeton~\autocite{Cosden2022a}.

[SuccessStory]
Founded in January 2017, the Research Computing department of Princeton University has experienced a tremendous growth from the initial two FTEs to a total of 18 FTEs in the span of five years~\autocite{Cosden2022a}.
This growth is based on a continuous influx of new funded projects once successful projects showcase the additional value of RSE services to researchers.

[Success Story]
The University of Manchester Software and Data Science group has successfully established specialised development services within their institution:
The “Mobile Development Service” \autocite{manchester_mobile} team consists of RSEs that focus solely on developing and deploying mobile apps.
Without a central RSE unit to anchor such specialised expertise, it would be difficult to establish such a service.
Also, having this expertise centralised allows for synergies in the deployment procedure for mobile apps:
The RSE unit can create institutional accounts with the app stores and manage the time consuming deployment process including hard-to-setup procedures like code signing.
Besides the technical benefits of this central deployment procedure, the institution will also benefit from the increased visibility and potentially be able to build a brand with its technological output.

[Success Story]
The Scientific IT Services of ETH Zurich (SIS) started in 2013 with a handful of members and has (as of March 2024) around 50 members.
In addition to HPC services, the group also offers RSE services in various areas.
These include the development of software applications for RDM, support in the development and improvement of scientific software or the long-term maintenance of software developed in research groups.
In addition, SIS offers services in the areas of data science, machine learning, bioinformatics, trusted compute environments, and training and consulting.

\subsection{Module 4: Teaching Services}%
\label{sec:teaching}

A central RSE unit can provide or organise training for researchers and RSEs in spokes.
This can replace self-education for foundational software development skills and provide a basis from which researchers can continue to learn more specialised skills guided by experts of the central RSE unit.
Since teaching material for foundational software development skills is freely available,
the tasks remaining for a central RSE unit are to adapt the material to local requirements as well as to organise and hold courses and workshops.

For more complex software development projects, a central RSE unit can offer individual training.
In both cases the expert RSEs from the central RSE unit can pass on their knowledge precisely adapted to the concrete needs of those that they support.
Finally, in organisations that educate RSEs, members of a central RSE unit may contribute to the curriculum of the institutional study programmes and teach corresponding courses. 

\subsection{Module 5: Create a Network of Institutional Partners}%
\label{sec:partners}

Within a research institution, a lot of groups or departments touch the topic of research software one way or another.
However, their coverage of RSE-related needs of researchers is often limited and their main responsibilities, even as diverse as they are across institutions, typically lie elsewhere.
While this is one of the main arguments for the creation of dedicated RSE units, it also shows the necessity for an RSE unit to closely interact with its respective partners.

In the following, we describe groups or units that can typically be found within academic organisations,
what they usually focus on and how we envision collaborations with an RSE unit.
However, note that as research organisations can differ widely from one another, so can the tasks and even existence of the entities below.
Arguments and conclusions below have to be adapted to specific circumstances when applying them to specific environments.

All research institutions in Germany can use the services provided by a central \textbf{IT unit} in one form or another.
The central IT unit typically looks after common compute and storage infrastructure in data centres and associated network infrastructure.
They also provide software services such Email, web services, databases and administer standard desktops and laptops (at least for administrative staff).
However, research software often has to work within the environment provided by the IT unit.
A central RSE unit can help researchers adapt their software to run on central services where necessary.
RSEs can also work with central IT staff to provide IT infrastructure well suited for research projects.
Usually, this requires a level of engagement and understanding of both the underlying research concepts and IT infrastructure that the staff of the IT unit or the researchers cannot provide alone.

If available, a second important partner is a scientific \textbf{library}, which has already gained tasks much beyond the preservation and organisation of publications on physical paper for quite some time.
Besides digital forms of rather traditional publications, these more and more include digital data and recently also software publications, their discovery and citation.
With the dedicated help of RSEs, research software can be enabled to be added to the organisational bibliography, facilitating internal reporting.
At the same time, through collaboration with the library, the RSE group can address the first two letters of FAIR:\@ Findability and Accessibility.

Both RSE and RDM units provide research-oriented services that are supporting digital research, covering different aspects of the research cycle.
They need to collaborate directly because research software both consumes and produces research data.
Requests for support on digital research therefore often touch both aspects, RSE and RDM\@.
Thus, a close collaboration between RSE and RDM groups helps everyone: both RSE and RDM groups by being able to offer a more comprehensive service than when working alone, as well as the researcher, who benefits from receiving this single coordinated service, instead of dealing with two independent entities.
The question whether RSE and RDM should be located in two separate groups or should be combined in one common group is intentionally left open, as the answer depends on local, pre-existing circumstances.

Some research institutions might host a dedicated \textbf{HPC group} which may or may not be part of the central IT unit.
HPC is an RSE-related field, so HPC groups might already provide training, consulting and funding opportunities in this area.
At the same time, HPC by nature focuses on highly efficient, many-core, if possible parallel computations.
The challenges of an average researchers often start a long way before reaching that level, and they never might need to.
There are obvious reasons to closely collaborate on both consulting and training, yet at the same time a central RSE unit has to provide a much broader portfolio.

\subsection{Module 6: RSE Infrastructure Provisioning}%
\label{sec:infrastructure}

IT and (potentially high-performance) computing infrastructure provisioning is usually the purview of an institution's IT unit.
However, a central RSE unit can provide extra services by acting as an intermediary for RSE infrastructure and by hosting pilot instances of new tools and services.
IT departments typically only provide the service for hosting and accessing IT infrastructures, such as RSE infrastructures.
Central RSE units are a link between the central services offered either by IT units or over-archingly available services on one side,
and RSEs in spokes on the other, offering documentation, training and best-practices to efficiently and effectively use available services and comply with established processes.

Furthermore, the central RSE unit can offer consulting for RSEs in spokes to guide selection processes of the tools and services best suited for each project.
This holds for existing RSE, or more general IT, infrastructure.
However, as scientists are working, by definition, at the cutting edge, they will often need or want to use the newest tools.
When such a need is identified in the course of a consultation, a central RSE unit can set up and provide access to pilot instances to evaluate these tools.
This evaluation will specifically consider a wider applicability of the tool, with the aim of handing over administration of widely required tools and services to, \eg{}, the central IT unit.

It is crucial that the RSE unit does not compete with the IT unit, nor should it duplicate existing infrastructure.
On the contrary, the RSE unit should act as a multiplier for the RSE-relevant services offered by the IT unit, helping RSEs to discover and use existing and upcoming services.
Similarly, the RSE unit can promote the use of the available computing infrastructure provided by an IT unit, helping with the support of the users when RSE-related questions in this context arise.
Once the mutual collaboration between an RSE unit and an IT unit has been established, a stricter policy-based involvement of the RSE unit for infrastructure requests is envisioned.
Overall, by acting as an intermediary for RSE infrastructure related requests, the central RSE unit can augment the central IT unit, providing RSEs in spokes with the specific support they require.

\subsection{Module 7: Research Software Engineering Research}%
\label{sec:rseresearch}

If software engineering research about research software is conducted at the research institution, an RSE unit can serve as a valuable resource and experimentation field to these researchers.
It might therefore be of mutual benefit to co-locate SE researchers and RSEs.
Additionally, RSEs employed at the hub can be given the opportunity to conduct research on meta aspects of RSE work and publish about them.
This allows the staff working at the hub to contribute to and shape the emerging field of RSE research.

\subsection{Module 8: Software Maintenance Service}%
\label{sec:maintenance}

Funder policies such as~\autocite{dfg_gsp} require long-term preservation of used research data and software in an adequate way.
For research data, established procedures to implement this requirement exist.
For research software, dedicated archiving solutions such as Software Heritage~\autocite{DiCosmo2020,DiCosmo2023} or Zenodo's GitHub integration~\autocite{GitHubZenodo} exist.

In contrast to research data, the long-term availability and usability of research software requires more than an adequate archiving method:
Software maintenance is the ongoing change process of software after its release.
It includes  both fixing bugs that are discovered in the software and adapting the software to changes in the execution environment such as hardware, operating system, toolchain and software dependencies.
In the scientific community there is a demand for long term maintenance of research software,
but academic labour turnover and missing funding schemes make research software maintenance often rely on the (potentially unpaid) efforts of individuals.

An RSE hub with long term core staff can partially solve this problem by taking over maintenance tasks.
In order for this to be feasible two criteria need to be met:
\begin{itemize}
\item The software needs to be developed according to software engineering best practices with a strong emphasis on testing and continuous integration.
\item The RSE hub needs to be involved during the development period either through development or consultation services in order to ensure that best practices are followed and the required knowledge is transferred to the hub.
\end{itemize}

\subsection{Module 9: RSE Outreach}%
\label{sec:outreach}

One of the central tasks for the RSE unit is to connect the local RSE activities and RSEs to regional, national and international initiatives.
This can be achieved by contributing to events, position papers and the initiatives themselves,
either directly from the RSE unit or by advertising at the institution and matchmaking with local RSEs interested in becoming active beyond their current tasks.
The RSE unit thus contributes to the RSE communities on a regional, national or international level on the one hand and opens these up to the local RSEs and enables networking on the other hand.
It organises the bidirectional exchange between the local and the global community and is the central hub for information coming both ways.
Furthermore, the RSE unit should advocate the use of RSE techniques and best-practices within their institutions actively to strengthen the local community and to reach out to new groups whenever possible\todo{magi: this could be strengthened by invoking the hub and spoke model again}.

\section{Existing Implementations}

\begin{figure}
\centering
\includegraphics[width=\textwidth]{./group_composition_plot/group_composition_plot_the_fantastic_four.pdf}
\caption{National and international examples of RSE units and their service portfolio: Heidelberg and Princeton offer development services, whereas Jena and Reading focus mostly on teaching and consultation services.}%
\label{fig:survey}
\end{figure}

A number of successful RSE units have already been established in Germany and many more exist in other countries, especially the UK and the US\@.
In order to understand the service portfolio of these existing RSE units, we conducted a survey that received a total of twelve responses from Germany, the UK and the US\@.
We asked RSE units for the composition of their service portfolio --- the results are shown in Figure~\ref{fig:survey}.

From the gathered data and the additional free text information of the participants we conclude that the service components that we have identified in Section~\ref{sec:vision} are indeed relevant for existing RSE units.
Additionally, we see a large diversity in the weighting of these components, which is to be expected given the different environments of the RSE units.
Within this diverse data set, we identified two rather different archetypes of RSE units: those that offer development services and those that do not.
The RSE units offering development services would typically invest a lot of their resources into this component, whereas others put a much larger emphasis on teaching and consultation services.
We should note however, that our survey did not collect information about the size of the RSE unit.
It is likely that the RSE units offering development services are also larger in size,
and that their total resource commitment to teaching and consultation services is similar to that of those RSE units that do not offer development services.

When setting up a new RSE unit, it is important to find the best service portfolio composition for the local environment.
This depends on the demand by scientists at the institution, existing structures and the available funding.

\section{Realisation Strategy}%
\label{sec:realization}

We propose building blocks for individual realisation strategies for central institutional RSE units.
We start by listing different possibilities for funding RSE positions at a research institution.
Following that, we describe a potential transition pathway, starting from existing structures that have grown in research alliances such as \eg{} DFG-funded Collaborative Research Centres or also in research departments of an institution.
This is complemented by discussing the opportunity of outsourcing RSE services and the challenging task of identifying and hiring suitable RSE candidates.

\subsection{Funding Possibilities}%
\label{sec:funding}

We see four basic options for financing RSE positions, which we will briefly explain below:
\begin{enumerate}
\item ordinary budget positions,
\item the overhead of externally funded projects,
\item explicitly requested person-months in externally funded projects, and
\item dedicated RSE calls.
\end{enumerate}
While each option stands for itself, in reality, an institutional RSE unit will most certainly finance its staff by an appropriate mixture of possibly all four options.
The mixture at a particular institution depends heavily on the local conditions.

At research institutions, it is important to resolve the conflict between time-limited research funding and the need for permanent positions in order to be competitive with industry.
Experience is also an essential component of software engineering, which makes long-term employment indispensable.
In principle, pooling of positions and funds makes it possible to finance permanent positions from changing and mixed sources.
An institution’s leadership has to justify taking the corresponding risk of failing to raise external funds.
\begin{enumerate}
\item It seems natural to allocate ordinary budget positions for RSEs.
      However, particularly at German universities, it is usually impossible to create completely new budget positions and the only feasible way is to rededicate an existing position after its corresponding holder has left.
      Depending on the local circumstances at an institution, this can be a cumbersome and time-consuming process for each such position.
\item Funding organisations additionally allocate a portion of the direct project funds as overhead, which is typically divided between the institution and the applicant.
      We propose using a small percentage of the overhead agglomerated at the institution to permanently finance central RSE positions.
      Assuming a third-party funding income of EUR 50 million annually and a 20\% overhead, EUR 100,000 in permanent funding requirements for one person-year would only account for 1\% of this overhead.
\item In project applications involving the development of research software, corresponding person-months should be applied for to finance RSE tasks.
      In this way, an applicant can book a fixed number of working hours from the RSE pool and pay for the costs accordingly.
      This model has been successfully implemented at several UK universities.
      In order to scale, it needs to be supported by an institutional policy.
      Large scale collaborative projects can often apply for dedicated technical support positions that align well with the idea of RSE units.
      For example, the German Research Foundation allows requesting funding for “central service units or external service providers” in grant proposals aimed at developing research software\autocite{katerbow_dfg_rse_2024}.
\item Funding organisations are increasingly recognising the need for sustainable research software
      development and are setting up correspondingly designated funding programmes.
      The DFG has already organised three calls for proposals in 2016\footnote{\href{https://www.dfg.de/resource/blob/172674/1bcb181a6451fdac9d94421776b52798/161026-dfg-ausschreibung-forschungssoftware-de-data.pdf}{Nachhaltigkeit von Forschungssoftware}}, 2019\footnote{\href{https://www.dfg.de/de/aktuelles/neuigkeiten-themen/info-wissenschaft/2019/info-wissenschaft-19-44}{Qualitätssicherung von Forschungssoftware durch ihre nachhaltige Nutzbarmachung}}, and 2022\footnote{\href{https://www.dfg.de/en/news/news-topics/announcements-proposals/2022/info-wissenschaft-22-85}{Research Software – Quality Assured and Re-usable}}\autocite{katerbow_dfg_rse_funding_2018}.
      It is to be expected that even more programs will be launched in the future.
      An already established RSE unit at an institution increases the chances of being successful in such calls.
\end{enumerate}

In addition to these funding options, we encourage funding agencies to provide seed funding for the establishment of RSE structures.
Such seed grants ease the decision making process in organisations and give RSEs the leeway to establish collaborations with researchers without the direct need to ask for remuneration.

% We see two possibilities to acquire additional funds.
% Some can come from the convergence of existing central structures.
% Other funds must come from the research departments.
% One possibility is that a share of all overheads goes to the RSE unit.
% This solution is simple in principle and also immediately makes the RSE unit known, but will also require effective reporting to determine the required share.
% However, it might be difficult to realize the required reallocation of resources.
% Another possibility is the pooling of RSE funds from different research departments, which brings its own administrative burden, and is probably not generating long-term positions.
% To generate these RSE funds it is required to obtain them from external funders like the DFG, a strategy that can be supported by an institutional policy to ask for these.
% This mixed funding scheme brings us to organisational topics that facilitate the money generation.
% From the side of the funding agencies this can be supported by requiring a digital support strategy from proposals similar to RDM plans.
% If this strategy/policy is followed institution-wide, we believe that a consistent stream of income can be generated that enhances the bare-level support that is generated by the fixed funding from the central budget.

\subsection{Transition Pathway}

We present a possible transition pathway from RSEs distributed over an institution and associated purely with research working groups and corresponding projects towards an institutional RSE unit.
After starting with initial measures not necessarily requiring dedicated funding, we discuss developing a business plan and how the RSE unit can be established.
We conclude with measures for promoting its growth.

\subsubsection{Initial Measures}
The following measures initialise the two modules presented in \autoref{sec:network} and \autoref{sec:teaching}.
While dedicated funding certainly is beneficial already for this step, it is not strictly necessary.
Once the two measures are in place, they can be used to illustrate the need for institutional RSE activities and therefore support funding proposals.

\paragraph{Network of RSEs}
Forming a network of RSEs localised at an institution can be initiated by any existing RSE individual or group that is preferably already in contact with other RSEs at the institution.
An institutional dedicated mailing list, chat group and possibly other communication platforms can be created and a request for participation can be circulated via institutional channels such as an employee newsletter.
First common events such as social gatherings or RSE-related seminar talks can be organised and announced via the communication platform.
If the initiative is geographically local, this process can be accompanied, facilitated and strengthened by founding a local de-RSE chapter\footnote{A list of existing chapters: \url{https://de-rse.org/chapter/}.}.
Such network-building has been successfully initiated and implemented at several German research institutions such as the German Aerospace Center and the Forchungszentrum Jülich.

\paragraph{Pooling of existing teaching materials and training offers}
Depending on local RSE efforts, teaching materials and associated training formats are likely to already exist,
distributed over individual institutional groups.
With the established network, the materials can be pooled and joint training can be offered to a wider institutional audience.
This step can be facilitated and formalised by offering introductory courses with a recognised curriculum as provided by The Carpentries\footnote{Examples \url{https://software-carpentry.org/lessons/}}
or CodeRefinery\footnote{Examples \url{https://coderefinery.org/lessons/}}.

\subsubsection{Developing a Business Plan}
Decision processes at institutions usually require a business plan on which to base the decision on whether to establish an RSE unit.
The business plan should include an outline of the RSE unit, its responsibilities and offerings as a subset of the nine modules presented in \autoref{sec:vision}, the funding, and the resulting benefits for the institution and its researchers.

A rather difficult and crucial question can be the positioning of the RSE unit within the organisational structure of the institution.
A canonical place would be a new subunit of an existing unit close to software,
training services and computing such as the local or central IT unit or the library.
Since most institutions already have an RDM unit, it seems natural to add the RSE unit as a parallel structure.
Another choice for the parent unit, particularly at universities, is the faculty for computer science.
Determining the best place may involve discussions with several stakeholders at the institution and can already be beneficial for creating a
network of institutional partners, see the module described in \autoref{sec:partners}.

The business plan also needs to address funding for the RSE unit's initial staff.
We consider it necessary that there is a certain amount of base funding provided by the institution that covers a basic RSE unit because much RSE work is not project based.
While options can be drawn from the discussion above, specific ideas should be discussed beforehand with the decision makers.
In order to facilitate long term growth of the RSE unit, an institutional policy for requesting person-months in externally funded projects dedicated to RSE should be established.

Another part of the business plan should be the governance structure of the RSE unit.
One of the decisions to be made is if the unit head is supposed to be part of the unit itself or if the unit will be headed by somebody outside of it.
Additionally, installing an advisory board can be proposed in the plan, recruiting from the prospective institutional partners.

\subsubsection{Establishing the Department}
Once the business plan has been approved by the institution, the RSE unit can be established accordingly.
The initial staffing depends crucially on the local institutional conditions.
One promising possibility is to start with two positions.
The first position is an RSE coordinator who is the contact person for all embedded RSEs.
Among other things, they are also responsible for organising meetings, developing training programs and reporting to stakeholders.
The second position is a central RSE, responsible for providing selected services and infrastructure.
These central positions complement the network of RSEs as described in~\autoref{sec:network}.

Drawing from the business plan and considering the actual initial staff situation, a first task of the centrally funded structure is to define a basic service portfolio according to the modules described in \autoref{sec:vision}.
In addition to the already mentioned networking and teaching, see~\autoref{sec:network} and \autoref{sec:teaching}, it seems natural to start with consultation, see~\autoref{sec:consultation},
as this allows to evaluate the potential necessities for other services such as development, infrastructure provisioning and maintenance.
An extension of the initial service portfolio for a larger target audience requires the acquisition of funding for further positions, see below.

\subsubsection{Growth of the Department}

\paragraph{Acquisition of further funding}
The most promising possibilities to acquire further funding seem to be explicitly requesting person-months in externally funded projects and dedicated RSE calls, as discussed in \autoref{sec:funding}.
We believe that the credibility of a research proposal that is asking for RSE funds is greatly enhanced if the RSE unit is part of this proposal from the beginning.
Over time, more and more researchers and proposals are expected to follow the institutional policy such that a consistent stream of income can be generated.

\paragraph{Offering of additional services}
With the additionally acquired funds, the service portfolio of the RSE unit can be enhanced regarding the modules described in \autoref{sec:vision}.
Obviously, the selection of modules and their share in the overall portfolio depend on the services that have been applied for in the corresponding proposals.
As this is strongly connected to hiring persons with the required expertise, this has to be carefully planned, see also below.


\subsection{Outsourcing}

Another possibility for the realisation of local RSE Service providers is by forming a spin-off and pooling the RSE Skills into an external company, which has benefits but also drawbacks.
This is an idea, that so far lacks examples but for completeness we list some of the advantages and disadvantages.
Among the most obvious benefits is that this enables the creation of contracts outside of the WissZeitVG.\@
This also widens the customer base of the RSE unit since the newly founded company may obtain contracts from industry.
If this company is backed/branded by the institution, this enables another possibility for an institution to interact with local companies.
This might open opportunities for employees to more freely move between industries and academia.
But there are drawbacks.
Since the company is now an entity external to the institution, the Vergabe-Richtlinien have to be fulfilled, which could \eg{} mean to publicly invite tenders in order to have a competitive procedure.
A company has to be profitable entity but this could be partly softened by founding a not-for-profit entity.
Moreover, during the outsourcing contract, there has to be a coordinator at both sides and the flow of information from the academic institution to the contracted company has to be established.
These are some examples of additional administrative overhead due to the interaction with external partners, and
certain domains will have issues with ensuring data privacy in this collaboration.
Some of these issues might be legally alleviated by forming framework agreements between an institution and the company.
It is of particular importance to agree on requirements and handover criteria, including quality assurance and license specifications.
On top of these drawbacks, there are soft factors, like whether an external company is accepted by scientists.

\subsection{Staff Acquisition/People}

RSE units need to be staffed, but where do potential employees come from?
So far, researchers accidentally find themselves in the role of an RSE because they pursued software development as part of their research.
A more deliberate approach with specific RSE education may be necessary to train people in sufficient numbers for central RSE units.
Being an RSE should be a career worth aspiring to, just as any other profession, with a long term perspective.
This is a topic covered by a separate paper \autocite{goth_foundational_competencies_2024}, but we provide a brief summary here:

These RSEs will bring a diverse set of skills centred around the topics of research,
digital tools, and team-based work and hence can easily offer the consulting services mentioned in the previous section and guide people to their implementation in their workgroups.
To fill gaps, the RSE unit can also maintain a roster of freelance workers.
In order to retain RSEs it should be possible for them to become experts in a field and hence this should make this job more attractive to budding RSEs,
in order to mitigate the problem that some will only see this job as a one-year stint after their PhD and then move on to something else.
To facilitate the retention of skilled people, industry has long identified education as an effective tool.
For RSEs, this should be helped by yet to be formed academic facilities that enable them to keep on learning skills after their first professional qualification, supported by the respective certification programs.
In the longer run, Research Software Engineering should be integrated into the existing study programmes.
One option here is the creation of an RSE master as a specialisation for computer science or application-domain bachelor programmes.
This should be complemented by adding a minor in these application-domain study programs such as biology, music, engineering etc.\ to facilitate the communication between the corresponding two groups of RSEs.
There are already some master's programs available, (\eg{} in Berlin, Munich and Stuttgart) that develop this specialisation on top of a domain bachelor.
Moreover, there are data science curricula in the process of being created.
A curated and continuously updated list of these programs is available at~\cite{learnandteachlearn}.

% on the topic of acquiring and retaining staff, there's \autocite{VanTuyl2023} from US-RSE discussing career pathways, and \autocite{wright_ambivalent_internal_consultant_2009} about internal consultants and their struggle coming to terms with their work identity (in the management field)

%\begin{thebibliography}{9}
%\end{thebibliography}
\printbibliography[heading=bibintoc]

\end{document}