DRAFT: Fedeproxy will reuse the Gitea codebase

Early 2021 I started to work on fedeproxy, to build communication bridges between forges. It would allow to create a bug report on a project hosted on GitLab even when using Gitea. At that time I knew very little about the ActivityPub protocol, the Gitea codebase or its community. But I thought it would be years before all forges are able to communicate with each other using some kind of standard protocol and the past months confirmed this initial impression. So I decided to bootstrap the codebase of fedeproxy using the language that I’m most comfortable with, Python.

During the summer of 2021, while working on the fedeproxy python codebase, I got a better understanding of what is at stake when implementing ActivityPub. I also got a chance to better understand how the Gitea community is organized. The work I did on a grant application to implement federation in Gitea was an opportunity for personal interactions with a few of the most active members of the Gitea project. Since July 2021 I also became a co-administrator of the Chapril Gitea instance.

In August 2021, I started to doubt that pursuing the implementation of fedeproxy in python was the best option. There was no push back, on the contrary. A month later I decided to flip the switch and Pierre-Louis also decided to go in the same direction.

In retrospect it was a mistake to begin with a python codebase. But I did not know then what I know now. It would be an even bigger mistake to keep going with the current codebase for the sole reason that it exists. It would deprive fedeproxy of the following benefits:

Instead of bootstrapping an entirely new codebase, I decided to start with a Gitea fork which yields additional benefits:

  • Dump/restore feature;
  • Migration feature which includes API interactions with a range of forges;
  • Embryo of an upload feature;
  • Mirroring infrastructure, although limited to git repositories;
  • REST server;
  • In memory representation of issues, pull requests etc.;
  • Release process;

A large part of the Gitea codebase is irrelevant in the context of fedeproxy and the scope of the fedeproxy project does not fit in Gitea. For this reason it will remain a fork which may be challenging when the code architecture changes. But it will sometime be beneficial. For instance the integration of go-fed should be almost identical in fedeproxy and Gitea. Whatever improvement is implemented in fedeproxy can be contributed to Gitea, and vice versa. An example is the recently added private key generation.

I’m not aware of a software project that went this way and unexpected difficulties will arise. But I think it makes intuitive sense because fedeproxy’s ultimate goal is to disappear when all forges are able to communicate with each other. For instance, when Gitea is able to gracefully allow issues across instances, fedeproxy will become redundant for this particular feature and the associated code can be removed. The same will be true for GitLab when it natively allows issues to be federated between instances using a standard protocol and vocabulary. And when all forges use the same standard to federate issues, the fedeproxy code bridging issues can be removed entirely. Another example is the implementation of the ActivityPub protocol in fedeproxy: it will evolve at a different pace than the ActivityPub protocol in Gitea because the two projects do not have the same release cycle. But there is no reason to integrate the protocol in different ways and eventually fedeproxy will re-use whatever Gitea instead of have its own, slightly different, implementation.

@pilou this is essentially a summary of the discussions we had over the past two weeks on this topic. It would help a lot if you could write down the objections you raised regarding the switch to Go/Gitea: they were valuable and I may have missed a few :blush:

I am afraid I do not understand the “disolve into Gitea” part fully. Do you mean that developing something for e.g. Gitlab and other forges is no longer part of FedeProxy, or that that will be in a wholly separate fork of Gitlab, or - maybe - once FedeProxy is a mature project you’ll just have solid specs for forge federation and any forge can grab it for themselves to implement stuff (Gitea being a reference implementation)?

1 Like

You are correct, this is ambiguous and incorrect. Fedeproxy may be a transparent proxy for Gitea and no longer useful for Gitea. But it will be a while before it becomes redundant for GitLab and in the meantime it can’t be “disolved”. I liked the image but it is confusing and needs rewording.

1 Like

I updated the description and title for clarity @aschrijver, what do you think ?

I am afraid still not clear enough. But it may be because I am insufficiently in the loop of project objectives and scope. What I think / deduced:

  • What: Federated issue management (MVP?)
  • How: Standard protocol and vocabulary (data format)
  • Where: Forge-specific implementations (?), starting with Gitea (Golang)

Gitea is fully committed to adding federation support, which is great. What purpose serves a full fork that you need to keep in sync and where FedeProxy federation has subtle differences with Gitea federation?

I would understand this fork if FedeProxy were an independently installed application that you deploy side-by-side with a forge product and which communicates via API’s. Then nothing will dissolve over time, however. But this would make sense to me: you bootstrap with Gitea, strip what you do not need, and build upon what remains. It would also allow a self-hosted FedeProxy-only instance (in the sort of forge-specific Ports & remote Adapters architecture we discussed some time ago). It would be a reference implementation where best-practices for protocol and vocabulary is used and with full documentation and tests accompanying it.

In this setup the “where” would be:

  • Where: FedeProxy module, independently installable, optionally self-hostable (and reference impl. for forge devs)

Another reason for a fork (with the original “where”) might be that the MVP of Gitea would be different in requirements and use cases than the MVP of ForgeFed while protocol and vocab are evolving, but would ultimately be integrated into Gitea, after which Forgefed ‘dissolves’ their Golang impl. and move on to the next forge federation support quest.

Yes on the What and Hown. No on the Where.

It is.

Here is an example based on what will happen:

  • fedeproxy merges the add user settings key/value DB table pull request in its codebase
  • fedeproxy implements the ActivityPub’s endpoint that returns information about an actor, including its public key (e.g. /users/loic), based on the code from the “add user settings key/value DB table” pull request
  • a pull request is opened in Gitea to propose the code implementing this new endpoint
  • the pull request is merged into Gitea
  • fedeproxy merges back the Gitea codebase and there no longer is a difference between Gitea and fedeproxy on this particular feature.

This is what I meant by “disolution”. It would of course be much, much less work if the endpoint could be implemented directly in Gitea instead. But the lifecycle of fedeproxy and Gitea are different and it would block any progress on fedeproxy during months while waiting on Gitea.

There are part of fedeproxy that do not make sense for Gitea to include, such as acting as a reverse proxy for the ActivityPub protocol on behalf of a GitLab instance. This will not be “disolved”.

It is very close to what I have in mind, with the following difference:

  • Do not strip the uneeded code, just ignore it
  • Merge the Gitea codebase on a weekly basis in fedeproxy
  • Build fedeproxy specific code in a way that minimizes the likelyhood of a merged conflict

Another example of something that is not in scope for Gitea but is supported by fedeproxy: Mercurial. Whatever code deals with Mercurial won’t be contributed back to Gitea. Not that it’s impossible in theory but the likelyhood of that happening is very low in the next two years.

I think I understand now. The FedeProxy fork (which could technically also be a Gitea branch if it wouldn’t have to address broader requirements for other forges) will gradually “meld” anything purely Gitea-related up to the point where Gitea has native “federation support”. At that point the FedeProxy Golang codebase will take on a life of its own, and become a more independent app that supports ever more forges and is installed side-by-side to them (like in the case of Gitlab).

When project “independence stage” is reached you may still incorporate Gitea code, but less frequently and there’s no need for the high-effort syncing anymore. Similarly Gitea from then on will evolve their own federation capability on their own, now based on mature standard specifications that evolve too.

If I am on the right track in my line of thinking, then eventually you’ll want to strip, rather than carry the ballast of ‘dead code’.

1 Like