State of the Forge Federation: 2023 edition

June 21, 2023

If you already read the “State of the Forge Federation: 2022 edition”, you may want to skip directly to the “Executive summary” section below.

Millions of Free Software developers forgot why it matters to own their tools. They know, better than anyone, how to fix and improve them. But when they choose to collaborate only via the most popular proprietary software forges, they are denied the right to use their skills and cannot work with fellow developers who are banned because they reside in the wrong country. They have been made to believe that the tools they use daily to craft their own software are out of reach. As if their software was a product that could be separated from the other software running the tests, allowing changes to be merged or bugs to be filed. But software is a process, and whoever controls it ultimately decides what the developers can do and how they communicate.

Some Free Software developers chose to use different tools, such as email and DVCS without a web interface, to collaborate in a distributed way. Many others decided to run their own Free Software forge and work in a decentralized way, improving their own tools and independently deciding who they work with. But even then, they are isolated instead of being federated with each other.

The ongoing forge federation effort is a simple proposition: online software forges must be able to communicate. It must be possible for developers to work together on the same software project regardless of the user interface they use. Just as it is possible for someone to send an email using their preferred client knowing the person receiving it will be able to read it and reply even if they are in a completely different environment. This will help developers to move away from software forges they are forbidden to change and regain control of their tools.

Is forge federation ready to be used by developers around the world? Not yet. But the first Forgejo release with native federation implementation based on ForgeFed and F3 is expected next year. As you read this “State of the Forge Federation”, you will get a high level view of what happened since June 2022 and where it is going. You will also realize that there is a lot of work ahead and this may motivate you to join the effort and help forge federation become part of the daily life of every developer.

Executive summary

Late 2022 Forgejo, a new forge with a focus on federation was created. Dozens of people contributed to its making and it is now used in production by ten of thousands of users at Codeberg, Disroot, etc. This large and unforeseen undertaking diverted the energy of most contributors set to work on federation features and rendered most of the 2022 predictions obsolete.

Since 2021 the strategy to advance forge federation was to have standalone proxies (forgefriends and ForgeFlux) to create bridges between forges that do not know how to communicate with each other, until they come up with a native implementation using ForgeFed, ActivityPub, F3, etc. Unfortunately for-profit companies never give priority to federation because it tends to lower vendor lock-in and the prospect of a native implementation was years away.

The apparition of Forgejo changed that timeline dramatically. It is developed by a community under the umbrella of a non-profit organization dedicated to the interest of the general public and federation was a priority from the start. Instead of installing a standalone proxy (forgefriends or ForgeFlux), developers will be able to install a Forgejo instance to mirror entire projects from GitLab.com to Codeberg.org. Or watch over a single issue from GitHub in the comfort of their preferred forge.

The ForgeFed ActivityPub extension made significant progress and the section about object capabilities is particularly noteworthy. Its reference implementation (Vervis) underwent an important refactor, switching to a networked actor model and a web app named Anvil was created to work with it.

The Friendly Forge Format came into existence with a draft specification and reference implementation for which a driver is developed in Forgejo. It also spawned the creation of a Wikidata project on forges for the purpose of keeping a durable inventory of software forges characteristics.

It is striking how much of the landscape changed within the past year, both in terms of projects and people involved. Through an unexpected series of events, the foundations on which forge federation is being built have shifted and there is reason to hope it was for the best.

By June 2024 it is expected that:

  • At least one forge software project (https://forgejo.org) will publish a stable release that implements federation
  • At least one public forge (https://codeberg.org) has federation enabled
  • The ForgeFed emerging standard defining the ActivityPub vocabulary for forges will have released its first draft and progress will be made on the Vervis reference implementation and Anvil, its web interface
  • The first stable release of the Friendly Forge Format, a open file format to represent software projects, will be published
  • The first beta release of the Starchart forge spider will be published
  • The number of people working on advancing forge federation will remain the same
  • Progress reports will be published on a monthly basis

What happened between June 2022 - June 2023

Software & Standards

Events & publications

State as of June 2023

Relationships

Here is how projects and standards relate to each other:

  • Forgejo (previously forgefriends) rely on a custom (based on go-ap) implementation of ForgeFed.
  • Friendly Forge Format (F3) is used by Forgejo for import, export and mirroring with other forges
  • ForgeFed is an ActivityPub extension dedicated to forge federation.
  • Anvil is a web client developed in close cooperation with ForgeFed and Vervis.
  • Vervis is a software forge meant to be a reference implementation of ForgeFed.
  • Starchart is a software forge spider designed for federated forges.

Software

Standards

Community

About ~five people are working daily on forge federation. The number of mentors and contributors involved is ~100. It is estimated that mentors have a strong impact on how forge federation moves forward.

What is likely to happen by June 2024

Software

  • Anvil
    • Makes progress
  • Forgejo
    • Communicates:
    • with Mastodon via ActivityPub
    • with other Forgejo instances via ActivityPub and ForgeFed
    • Mirrors software projects from Forgejo, Gitea, GitLab or GitHub
    • Published its first beta release
    • Publishes monthly reports
    • Is used daily by at least one person for its federation features
  • Friendly Forge Format
    • Publishes a stable release of its Go reference implementation
  • Starchart
    • Implements a UX that is good enough to make it usable to explore Free Software on indie forges
    • Supports crawling GitLab
    • Publishes crawled data using Git repositories instead of tarballs
    • Publishes its first beta release
  • Vervis
    • Makes progress

Standards

Federated forges

  • Public forges
    • Codeberg runs Forgejo with federation enabled

Community

The number of people working on advancing forge federation will stay the same. The availability of federated features that can be installed and used is expected to have a significant impact on the motivation of developers to participate.

Methodology, scope and definitions

Instead of this document, it would be better to create a roadmap. But how can that be done when the majority of the work is done by volunteers who are not constrained by anything except what feels right to them in the moment? Only when people working towards a goal are paid staff can they be assigned tasks to complete a roadmap. It would be tempting to create a roadmap that includes everything that needs to be done to further forge federation and, a year later, tick the boxes of what was done. This is however (i) extremely difficult because there are thousands of tasks that would help further forge federation, (ii) probably useless because hundreds of those tasks are likely to become obsolete within a year, (iii) there are so few people working on federation that only a fraction of those tasks will be completed in less that twelve months.

Instead of spending weeks (if not months) creating a large roadmap, a list of the goals that are most likely to be achieved in the year to come was created. The likelihood that a given goal is reached is an educated guess, at best. It is based on what happened in the previous year, the commitment people and organizations publicly declared to further forge federation and the gut feeling of the reviewers of this document. Hardly scientific, but maybe a useful insight for the reader.

Scope

To keep this document manageable (and updatable on a yearly basis) it has a limited scope. Anything that has not be active in over a year is assumed to be unmaintained (or discontinued) and not covered. The long term vision that drives people and organizations to contribute to the advancement of forge federation is an ongoing discussion that is also not in scope.

Only software explicitly dedicated to implement forge federation is in scope. For instance the implementation of go-ap is not in scope because it implements the ActivityPub protocol which has a broader scope than forge federation and it does not include anything related to forge federation. Other examples are software designed to interface with multiple forges, for bug reports (buggregator, git-issue, etc.), pull requests, … because they help developers overcome the absence of federation between forges. They are workarounds that would not be necessary if forge federation was available everywhere.

The same goes for standards: ActivityPub is not in scope but ForgeFed is.

Software and standards

A software is something that is designed to be installed, run and upgraded over time: it is not a product, it is a process. People involved in the making of a software may have different methods to achieve that. For instance developers who value user centric design will start with User Research before anything else. They will also figure out the best User eXperience before implementing any kind of UI. These various preferences regarding the software development process are not compared or mentioned.

A standard is similar in the sense that it must be maintained over time, updated and released.

Releases (software and standards) are named as follows:

  • Not released when there has never been a release
  • Developer release when it is only meant for developers and not users
  • Alpha when specifications, functionalities and interfaces are in flux but can be used for a test drive
  • Beta when everything is stable and only bug fixes and minor improvements are expected
  • Stable when nothing changes, except for critical bug fixes such as security issues for software or inconsistencies for standards

Publications

In some cases the work done in the context of a software project can be useful to other projects, even though it is not a release containing software. For instance, a User Research report focused on what developers need to understand when their intended user base is using multiple forges. Such a publication is mentioned at the date it is published. And again if it is updated at a later time.

Communities

A community is a group of people, either part of an organization or an informal collective. People involved in the making of a software project or a standard are a kind of community, whether they are volunteers or paid staff of an organization. Even if they do not identify themselves as a community, they will be perceived by outsiders as a group of people brought together because they have a common interest.

Although the governance of the community associated (implicitly or explicitly) with a software project or a standard has a high impact on how it is likely to evolve in the future, analyzing such a relationship would require an extensive, case by case, analysis and is not in scope of this document.

Some communities may exist to further forge federation without being bound to a specific software or standard. For instance, a community could be created for the sole purpose of organizing a webinar every year gathering all forge federation related projects or publishing this document.

Volunteer mentors

People talking together currently have a strong influence on how forge federation makes progress. As of June 2023, there are at most ten people in the world committed to create software or standards dedicated to forge federation. When they hit a roadblock, discussions with volunteer mentors can allow them:

  • to go in the right direction instead of continuing towards a dead end
  • solve a specific problem and keep going

When mentoring is not done on a volunteer basis, there are deliverables which are paid for and they can be accounted for just like modifications to standards or patches to software. That can be quantitatively measured in a way discussions between volunteers cannot. Here is an example: in October 2022 it was discovered that Gitea, the codebase on which most federation efforts and funding were focused, was taken over by a for-profit company. Forgejo was created to re-create a favorable environment, which is common place in Free Software. Nine months later it is a healthy project with a steady stream of releases. This success is the outcome of the discussions and advice from a dozen of mentors that lead to a sum of decisions that were good enough to create the right conditions.

The contributions of mentors to the advancement of forge federation is assigned a value that is agreed upon by the reviewers of this document, based on their intuition:

  • minor when mentors hardly have a visible influence
  • significant when there is a general consensus that mentors have a positive influence
  • strong when mentors are a driving force without which progress would not happen

Forge federation is volunteer driven

People contributing to forge federation are predominantly unpaid volunteers which makes it a community effort and it can be observed (like in Wikipedia) that most of the work is done by a minority of the participants. It is estimated that around five people are working on forge federation on a daily basis and no more than a hundred contribute occasionally, either by mentoring, reviewing contributions or proposing changes.

Monthly progress reports help newcomers quickly figure out what happened in the past and why, in a way that would be more time consuming if they only had list of issues to browse from.

Funding

Most of the globally available funding earmarked for advancing forge federation was not spent as of June 2023 (around 20K€ out of 150K€) because some beneficiaries did not claim it yet. At least one opportunity to apply for grants (between 65K€ and 600K€) did not raise much interest. Two developers were funded to work on federation full time and three other part time in the June 2022 to June 2023 period. Their work consumed less funding than what was globally available. This suggests that forge federation is likely to remain a volunteer driven effort in the next twelve months.

Search

Categories

Tags