On using part of the forgefed specification

Bonjour,

There is a debate about whether it is possible/useful to use part of the existing forgefed messages (specifically commit and ticket).

The fedeproxy project will use the commit & ticket messages for server to server activity notification, exclusively (no webhook or other mechanisms, only ActivityPub). But it won’t use it for synchronizing the forge content: this will be done via git/hg pull/push which is well suited to distribute a dataset that needs to be strongly consistent at all times. Both a complementary and there is no obstacle to using part of the forgefed spec in this way.

However, an objection was raised a few days ago on the libera.chat#peers IRC channel:

UserB: the short anbswer is that no one can implement the spec in any forge until the spec is finalized - o/c the work could begin before the spec is fininshed; but there is no reason to - after the spec is finalized, then it would be more important, for which forges people are implementing - im sure that will happen in time
UserB: for now, all that is important is for people to be working on the spec itself, and for others to remain interested until it is ready

In an attempt to figure out why, I posted this message today:

(12:34:26) dachary: short question: I’d like for Gitea to send an ActivityPub/forgefed/commit message (https://web.archive.org/web/20201030140622/https://forgefed.peers.community/modeling.html#commit) whenever there is a push to a given branch. go-fed already knows about forgefed. Has anyone tried that already?

To be continued

(12:34:26) dachary: short question: I’d like for Gitea to send an ActivityPub/forgefed/commit message (https://web.archive.org/web/20201030140622/https://forgefed.peers.community/modeling.html#commit) whenever there is a push to a given branch. go-fed already knows about forgefed. Has anyone tried that already?
(12:43:18) USERA: ehm: so, forgefed was started by fr33 who was also implementing it on its own forge called vervis. However I haven’t heard from him in a while. I ended up implementing federation on pagure and now I’m trying to update/finish the specification file.
(12:43:29) USERA: Pilou: are you the developer of fedeproxy?
(12:44:39) USERA: dachary: I’ve never heard of anything about forgefed on Gitea
(12:45:35) dachary: USERA: yes, Pilou is a developer of fedeproxy, I’m another one :slight_smile:
(12:46:46) dachary: USERA: here is the discussion regarding forgefed/ActivityPub/go-fed in Gitea https://github.com/go-gitea/gitea/issues/14186
(12:50:48) dachary: I discovered your work on pagure ( https://pagure.io/pagure-forgefed ) a month ago and that was very good news.
(13:11:14) USERB: fedeproxy sounds super interesting!
(13:16:44) USERB: so is it intended for fedeproxy to treat developement MLs as another forge that can be synchronized?
(13:17:05) USERB: i read something about a github<->ML bot in regards to kernel development on lwn.net, i’m wondering if it’s related (or could be)
(13:17:17) USERB: it would be nice to reconcile web/mail development ecosystems :slight_smile: :slight_smile: :slight_smile:
(13:17:27) USERB: how can we help the project?
(13:31:30) USERC: i suppose that depends which project - in the past hour, at least 4 projects have been mentioned
(13:35:28) USERB: explicitly fedeproxy :slight_smile:
(13:36:05) USERB: also dachary i noticed a question about funding on the gitea issue for activitypub/forgefed support… did your fedeproxy team approve of a budget for the gitea forgefed bounty?
(13:39:40) USERC: the general idea is that all forge-fed compatible servers/clients are equivalent - it is not important whether they are email, web, command-line, or anything else; any could be send and receive messages and commands transparently
(13:39:55) dachary: USERB: that is certainly in the scope of the fedeproxy project, yes.
(13:40:39) dachary: USERB: regarding funding yes, it is approved and ready
(13:41:12) USERB: USERC: yes but email servers don’t implement forgefed and github likely never will
(13:41:19) USERB: only web services can implement forgefed
(13:41:30) USERC: so the use-csae of bridging a mailing list to a bug tracker would be simply, one of the possibilities
(13:41:51) USERB: dachary: thanks so there’s 5000€ bounty that we can announce everywhere to try and recruit people to work on it? :slight_smile:
(13:42:21) USERC: if only web servcies could be forgefed peers, i would not be interested - that is selling the goals very short of its potential
(13:42:54) USERB: well you can have gateways/proxy but by definition only a web service can speak activitypub protocol (over HTTP)
(13:43:28) USERB: i was refering to this article about bridging ML/Github development: https://lwn.net/Articles/860607/
(13:43:38) USERC: forgefed is nothing but an abstraction layer, in order that it is irrelevant which software is involved
(13:43:49) USERB: it’s also a problem we’re facing for interop with the sr.ht ecosystem, really cool email-based forge
(13:43:53) dachary: USERB: yes
(13:44:34) USERC: github does not need to implement anything new - a forgefed compatibility bridge cuold be written for any forge with a complete API
(13:44:51) USERB: ah sure if you’re talking about a proxy, indeed
(13:45:14) USERB: nice for the bounty, i’m not interested in it ubt i’m sure folks will
(13:46:14) USERC: that is presumably what fedproxy will do - but no proxy should be needed to accomplish that - the highest goal that forgefed strives for is for everyone to be able to run their own forge, without depending on any other remote services
(13:46:43) USERB: posted on hacker news: https://news.ycombinator.com/item?id=27832020
(13:46:46) dachary: USERC: thanks articulating this. Do you see any reason why fedeproxy (which is a proxy between forges) could not use two of the messages defined in forgefed (commit/ticket)? I mean even while the spec is being developped, I see them as very useful.
(13:47:25) dachary: (and I concur on the ultimate goal! the ultimate goal of fedeproxy is to become redundant and disapear)
(13:47:54) USERB: dachary: i can at least see one very useful usecase (i started to write something like this a while back), to make forge webhooks forge-agnostic
(13:47:55) USERC: it seems to me that such a proxy equates to centralization
(13:48:25) USERB: so that you can process events from github/gitea/gitlab by parsing forgefed AS2.0 vocab instead of dealing with the subtleties of every forge
(13:48:32) dachary: USERC: how so?
(13:48:33) USERC: unless each user runs their own proxy, in which case its not really a prozy, and could simply be a feature of the local server/client
(13:48:59) USERC: it would be centralization if there were only one proxy server
(13:49:00) dachary: ah, yes, each forge runs its own proxy (not each user)
(13:49:37) dachary: it adds the federation features that are not yet native and runs alongside the forge
(13:50:04) USERC: ok i understand then
(13:50:39) dachary: since GitHub is likely to not ever do that, odds are there will be multiple proxy run by various organizations / people
(13:51:03) USERC: ideally though, most users would run their own forge - among the initial goals was that no one using the system, would ever need to visit the website of another forge
(13:52:01) dachary: that would be sensible
(13:52:14) dachary: this definitely what I would do
(13:52:46) dachary: this is definitely what I will do
(13:52:56) USERC: more preceisely: most users would run their own forge-fed compatible software - that could be a forge, or an email client, a CLI client, a GUI client, etc
(13:53:40) USERC: ie: the idea that email and forges wuold interoperate was primary from the start, just one of the infinite posibilities
(13:53:48) USERB: it makes sense, but each user requiring custom software makes adoption harder :-/
(13:53:57) USERC: other people wanted mastodon to be a full-fledged peer
(13:54:14) dachary: during the user research phase of fedeproxy the definition of a forge came up often. It looks like that’s fuzzy and encompasses whatever tools / methods developers use while working on software.
(13:54:21) USERC: yes, that is the standard webby appology
(13:54:40) dachary: how do you mean?
(13:54:44) USERB: dachary: yes a forge is a collection of tools to build more tools, web forges are only a category among those
(13:54:51) USERC: “do every thing in a web browser, so that people do not need to use native software”
(13:55:03) USERB: dachary: have you already contact libervia folks (goffi) to interop with XMPP federated forging?
(13:56:03) dachary: USERB: no, I did not
(13:56:11) USERC: its true, people like that, and it probably helps with adoption - in the long term, its kinda the thing that should be resisted though; because it is relying on someone else to operate your software for you
(13:56:49) USERB: USERC: well then we should go the P2P forge way with a gossip protocol like radicle.xyz
(13:57:08) USERC: the main reason why people would self-host a forge, is to avoid relying on a third party freebie service
(13:57:29) USERC: USERB: i could agree with that - git-ssb already exists though
(13:57:34) dachary: I dislike web based tools that are not self-hostable. I think it is not durable. I don’t use them also because I don’t use proprietary services anyways.
(13:58:10) USERC: if i werr to start forge-fed over from scratch, i would base it on git-ssb instead of activity put
(13:58:24) dachary: nice typo!
(13:58:47) USERB: last i checked git-ssb is not actively maintained nor used, and does not have integrations with other forging ecosystems :-/
(13:58:53) USERC: there a huge overhead to SSB though - the webby route is probably more palatable to folks
(13:59:15) USERB: i can’t say libervia XMPP-based forging is much more supported, but at least it’s selfdogfooded for libervia development
(13:59:35) USERC: USERB: so what? - if it works, it works - if no one is maintaining it, then someone can adopt it
(14:00:11) USERB: also i have opinions about the SSB protocol… it’s not THAT bad but it’s not as good as something could be done from scratch (JSON-inlined signatures, really??)
(14:00:13) USERB: sure thing
(14:00:29) dachary: there is one piece of software that every developer uses and that’s a real asset to implement federation: a distributed version control system.
(14:00:30) USERB: i don’t want to argue against anything, in the current state of thing any progress is huge progress :slight_smile:
(14:00:42) USERC: the point there, is that it does mostly everything that forge-fed needs to do already - if the plan was to start over fresh, most of the work is already done
(14:01:34) USERB: federated forging is not a hard issue, integration with other tooling is. that’s why forgefed is interesting to bridge the gap with the rest of the fediverse, in my view
(14:01:41) USERC: seriously, if forge-fed would abandon activity-pub now, and go the SSB route, it would actually be a huge advancement in progress
(14:02:54) USERC: all that would remain to do, would be to make existing forges like pagure, gittea, etc into git-ssb peers
(14:03:59) USERC: that would probalby be much more intrusive though - i doubt if any of the upstreams would be interested
(14:04:45) USERC: with activity-pub, at least it is webby enough, that existing forges may be convince to integrate support for it upstream
(14:05:34) USERB: i think that’s the point… it’s just a few more routes and some JSON to spit, compared to implementing XMPP/SSB forging
(14:07:32) USERC: yes, thats the same reason why email was not chosen as the communication medium - because that would require each forge to operate an email server
(14:08:12) dachary: there is something I’m not yet clear about regarding forgefed (be it ssb or activitypub based): it defines a model to convey information about events that are related to development (tickets, commits, etc.). Was there discussions about using forgefed to synchronize an entire development project from one place to another? i.e. migrating a project from a forge to another using only forgefed and nothing else.
(14:09:38) USERC: absolutely yes
(14:10:18) dachary: are there publicly available archives of these discussions so I can catch up?
(14:10:48) USERC: its not only about event notifications - anything that someone could do with a mouse on the web forge, they should be about to do from any client (email , command-line, whatever) without ever using a web browser
(14:11:12) USERC: yes there is a archive of the exporatory discussions
(14:13:05) dachary: yes, it goes both ways (notification and incoming messages that trigger a side effect)
(14:14:57) USERC: yes the full monte - most forges already support notifications as web-hooks
(14:15:02) USERC: i would actually start with this document https://notabug.org/NotABug.org/notabug-2.0
(14:15:36) dachary: I’m particularly interested in the discussions because ActivityPub (contrary to SSB or a DVCS) does not provide any kind of consistency. I have a preference for the strong consistency associated with a DVCS but migration of entire projects needs at least the eventual consistency that SSB provides (although it is challenging).
(14:15:38) USERC: that is the original general design goals before any specific tools/protocols were decided
(14:16:36) dachary: thanks, i’m reading that now!
(14:17:01) USERC: about a year later, the large “consortium” formed to have lengthy exploratory discussion, about the specifics
(14:17:18) USERC: that is this email archive https://framalistes.org/sympa/arc/git-federation
(14:20:37) USERC: that is where it was decided to use activity-pub, that it should be maximally generic, to support multiple VCS, arbitrary clients/peers, and so on
(14:26:15) dachary: (I’m a slow reader and I pay attention to every word, please bear with me :slight_smile: )
(14:26:22) USERC: it really depends what you mean by “the entire project” - FWIW, pagure already is 100% federated - it was even then 4 years ago
(14:27:09) USERC: with pagure, all tickets and documentation are git repos - so you kinda get federation for free with that design
(14:27:30) dachary: how do you mean 100% federated? One can interact with an issue located on a remote pagure instance from their own instance? that kind of federation?
(14:28:06) USERC: you may lose your gamified metadata, like “stars”, “followers” and such; but if you move the project to another server, you would lose them anyways
(14:28:07) dachary: ah, right: it’s an essential building block that all other forges (fossil excepted) miss
(14:28:38) USERC: yes you can clone the ‘tickets’ repo of any pagure project, commit to it, and push back
(14:28:56) dachary: with pagure one can sync the entire project using git and that’s an enabler for federation
(14:29:25) USERC: yes an enabler - you wuold need to write a few scripts to manage it
(14:29:49) ***dachary back to reading
(14:30:24) USERC: but it was no accident that git was chosen as the data store - those folks were thinking “outside the box” in that way - allowing for easy migration/interop
(14:32:51) USERC: so for pagure, forge-fed adds only that the other metadata in the DB (stars, forks, etc) could also be replicated/migrated/interop
(14:33:39) ***dachary nods and goes back to reading
(14:46:14) dachary: I’m done reading https://notabug.org/NotABug.org/notabug-2.0 and it is enlightening. I have many questions / discussions topics but … I’ll read the mail archive first as it presumably answers some of them already.
(14:53:08) dachary: I won’t read the mail archive: that’s too much :fear: :slight_smile:
(15:08:02) dachary: USERC: may I ask for a spoiler? How does it end? From the lack of posts I suppose it is no longer active and, as forgefed, asleep and waiting for someone to get to work and continue. How far am I from reality?
(15:49:03) USERC: yes there was a ton of discussion - it kept me busy most of that summer
(15:50:46) USERC: those discussion ended badly actually - the organizer bailed before the last remaining vote (which license) - abandoning the github repo at the same time, and leaving no one with credentials to maintain it
(15:51:57) USERC: but there was nothing remaining to discuss - even the license had been decided by consensus before the formal vote - so there was no reason to continue using that mailing list
(15:54:06) dachary: USERC: thanks for the short summary :slight_smile: