Ecobytes + allmende.io = ♥?

Compost
1

Fortsetzung des Gesprächs Social Intranet:

For Ecobytes and allmende.io, I could see myself running and maintaining the whole tool stack from the https://trello.com/b/mJsMLKTu/allmende-io board, but most important would be the tools offered right now:

via a Docker setup

  • Discourse (at least 2 GB RAM + 2 CPU cores)

the Node applications

  • Ghost Blogs
  • Etherpads
  • Federated Wikis

and further standard services with nice interfaces

Another important thing would be a GitLab / GitLab CI infrastructure.

As we’re living in times of perpetual betas and continuous lifecycle development within Legal Open Source Frameworks, I feel a strong responsability for the services I provide.

1 „Gefällt mir“
2

Hit together,

@almereyda & @gualter : I think the infrastructure discussion should be placed in a seperate thread. [Done, thx almereyda] But in short, my statement to gualter’s offer is “Why not?”

I think there’s nothing we could loose by accepting his generous offer. Ultimately we want to build a descentralized system and, for the moment we need at least 2 servers to achive that :slight_smile: It would be a great opportunity to check out various replication and rollout mechanism.

Back to the Projects posted by almereyda : I haven’t had the time to digg deeper into the single projects but I think some of them are very promising like

The offiria installation that samuel provided was only thought to be a intermediate solution until we have something more integrated up and running. But I think the topic is ‘closed’ for now as we commited ourselfs to use discourse and trello for the #14mmm communication.

I’d like to see a plattform which acts more like an aggegator (with the possibility to contribute) for specialized tools connected via a common metadata model. The base should be a stack of spezialized standalone tools and we have to provide a very simple API over the metadata model which makes it easy for us and others to ‘integrate’ their tools with.

This drastically reduces complexity for each contributer and makes it easy to integrate existing tools. From my point of view this goes into the direction of “Enterprise Application Integration” but with leaving out the “Enterprise”.

As I already told you in munich I think that the ‘new internet’ infastructure will be even more decentralized than it was in it’s beginning (with a few drawbacks inbetween). Every smartphone will become a server of its own and all of them are interconnected.

There is simply no need for a centralized server anymore since every device is connected to the internet all the time anyway.

1 „Gefällt mir“
3

There you go. It was already there. Next to Gualtar’s post you see where it has been referenced and from where it has been referenced. The same with the first post in this thread.

I just couldn’t find a function to split the post into two and then move only the part needed. Maybe someone should push this idea to upstream or check, if it already exists.

I have been working with the remoteStorage guys in the last months. I told you they are looking for developers. Their community was the place where I’ve found Discourse. The stack we could provide can grow quite quickly. Some milestones and additional partners would be nice. JPBerlin, IN-BERLIN, …? Who are the rogue hosters in Germany and Europe? I’m sure some italian anarchist collective must be out there.

You mean also for the #14mmm, right? Do you prepare a welcoming mail-pad, after my english resume of the workshop, that you could use as a blueprint, is finished?

So you’re basically looking for an integration platform as a service (iPaaS) like bipio maybe? Read more on that at the npm directory, too.When thinking of Semantics, the multi talented Virtuoso Universal Server appears here again.

When researching this, we’ll stumble across some flow-based approaches.

But tell me : why is it that most of the (free) Enterprise Systems I can find are written in Java? Nevertheless, as I’m a little GitHub-biased and like when code is there and up to date, it seems Mule or fabric8 are good candidates. But actually I cannot weight the choices at all. Some have at least OpenShift or Docker recipes.

Or we should build something simple with Node ourselves.

Regarding your decentralized every-device-is-a-server-idea you should read through the list of alternative internet projects again, if someone is not already working in that direction.

4

Wait; may it be that we are basically looking for Sockethub, which drives us back to the Unhosted / Offline apps movement again?

5

Hi @almereyda and @pierre_ozoux,

Although I like the concept of redecentralization, I set it up rather as a long-term target, rather than something we can reach with our immediate short needs. The idea of bringing Pierre here into the discussion was particularly to understand and plan together how:

  1. we can better adapt the existing infrastructure of Ecobytes (the 4 physical servers, which are currently undergoing revision/planning new setups) to use by the multiple services/applications the different collectives want to use
  2. find ways to articulate these different tools into one space with some form of integrated workflow for groups/organisations, particularly what we want to do with Drupal/Open Atrium 2 co-munity.net.

There is simply a huge need for such a thing and we have a lot of groups/users that are really waiting for the new co-munity to come into existance (many are already users of the old co-munity, which is far from being a decent set of tools, while others have used it/heard about and are waiting to bring their groups into the new tool). Not to mention that we need to have the new system

  • ready, optimized and with redundancy for the participants area of the huge degrowth conference - 1000-2500 participants,
  • interacting not only as general discussions,
  • but also doing open peer review,
  • collaborative document writing,
  • ideally (but not priority) also uploading media content.

Would be therefore great if we could provide a show-off of all these amazing tools that are out there and that we can service, already at the conference or in a near future. I propose for this effect that we have an Ecobytes/Allmende/hackers space at the Degrowth fair that will happen at the conference everyday from 17h-19h - anyone volunteering :wink: )

1 „Gefällt mir“
6

My suggestions would be as follows:

  1. Use a OpenStack IaaS with a OpenShift PaaS instances to quickly run VPSs, Docker instances or anything PHP / Node / Ruby … distributed on the four servers.
  2. Use a central, API-aggregating iPaaS like bipio or Sockethub. Nicely integrated into Drupal.

Therefore following the argumentation from the Seafile maintainers:

About the plugin system: Seafile will not have a plugin system. Instead, we will try to integrate other services by API’s. For example, we will not have a plugin for features like syncing calendar and contacts, but we will try to work together with system like Radicale (https://github.com/Kozea/Radicale). Many systems are focus on their specific tasks and do much better. So instead of replicate their work, it is much better to work together with them.

[via Google Groups]

I don’t believe in making OA2 an all-in-one tool, but in making it a flexible framework, where one can dock new Apps via their respective APIs.

My recommendations for the tasks are:

  • OpenStack + OpenShift for redundancy and scalability
  • Discourse for discussions.
  • Peer Review : should be something workflow based like Penflip or Authorea. Or something where one can comment on PDFs. Anyone seen this out in the wild already?
  • EtherPads + Fidus Writer for collaborative writing.
  • Media Goblin for media content. But that could also been done by ownCloud (slow!) or Seafile.
7

Pushing a bit the discussion again.

OA2 not as all-in-one tool but flexible framework - yes, definitely, it would just be too heavy and unreliable to have everything working on a platform. Furthermore, it makes totally sense what Seafile maintainers write.

A major aspect I think needs to be addressed is how the fine group structure+access control that of OA2 provides be projected into the multiple apps.

A simple short-term solution for implementing some apps is, like we are doing now with an etherpad iframe integration in OA, to simply index the random token from the URL part within a group - and then providing a listing for all the etherpads in the group.

I know little IaaS, PaaS and iPaaS. At what level would they work and articulate with the platform?

What about LDAP as an underlying auth system, keeping track of groups and users, on which other apps could also rely?

On peer review I already commented on the Trello board. I didn’t find any PDF commenting possibilities on the open world.

ownCloud was what we were thinking to provide, Seafile is new to me, looks interesting.

But, generally, how do we move forward? In particular, some of these things (peer review, etherpad) should be working by the end of May, to be used by 2000 Degrowth conference participants in their OA2 participants area. @almereyda would you like to directly contribute here, helping to make a few first steps for this framework?

1 „Gefällt mir“
8

I have been thinking a lot about this lately. You can see that in my writings here and in the Trello boards. Now it would be nice to have some other Ecobyterians joining the discussion.

I will now say a little to your most important points

  • Collaborative Writing
  • Peer Review

which both work very reliably. Further on I will go into some detail regarding

  • SSO/LDAP
  • ** aaS*.

Following will be a disambiguation between Ecobytes, allmende.io and co-munity v2.

Collaborative Writing

As you defined yourself, Etherpad Lite is the way to go. It’s a very easy installation process and a reliable service that can be extended by plugins. Which should be done for a professional Service like the one for Degrowth. I suppose the implementation time is around 2 to 4 hours.

If you want to have a more sophisticated approach that doesn’t lack a beautiful UI, please consider adding http://fiduswriter.org/ for real documents with annotations next to pad’ish scribbles.

Peer Review

As I have written in the co-munity v2 Board Peer Review Card Hypothes.is combines annotation with PDFs. I’ve now installed it and it works. Chrome users will have to install a custom-built plugin, that injects PDF.js into the browser, and/or the stock plugin (which is hard-coded at https://hypothes.is, which didn’t launch yet) + a bookmarklet. The bookmarklet works out-of-the-box in Firefox, as it’s using PDF.js internally already.

SSO/LDAP

Coming with Open ID Connect is the tighter integration of authentication and authorization for different web services. These layers can be built connected to some LDAP service, some have it built in. Please refer to the Trello Card about SSO again, where I’ve collected some implementations. I suppose the best would be to rely on the Gluu stack, as it’s already available.

Again, one would need a middleware (thanks @daniel :wink: ) that propagates the desired restrictions or permissions to the applications. Also not all applications have access-controls, like Etherpad. So you will have to manage your private URIs from the OA2 frontend solely.

I’m not sure if OA/Drupal is capable of doing so. But if you hard-wire your OA2 instance to the self-deployed Open ID Connect provider only, then people wouldn’t need to provide different login credentials for any service in your grid.

*aaS

This is the tricky question now, eh? :wink: Why I would also like to have Ecobyterians joining this discussion.

I have already laid out my strategy in multiple ways, but will try to resume again for easier comprehensibility. As I didn’t receive too much feedback on my work / comments / etc. on the Trello Boards, I have somewhat slowed down my productions. Otherwise you would now have access to more structural hypothesis images.

For detailed explanations, please also refer to your own research and as a starting point this Wikipedia article.

IaaS

Infrastructure as a Service means you can spawn virtual machines of different size within a federated cluster of servers. Or move them. Or move storage independent of CPU. Or allocate RAM dynamically. Or create VLANs. Or run docker images.

We would need those for big applications like Groupserver, GitLab, Discourse, database servers or Seafile.

PaaS

Platform as a Service means you can push application code (Node, Python, PHP, …) to your service and it will be a running program. This can be tightly integrated into Continuous Integration (CI) lifecycle processes.

This would include Etherpad, Fidus Writer, remoteStorage, Open Atrium, WordPress, …

SaaS

Software as a Service is the tools we want to provide in the end.

Now the tricky part is if and how Ecobytes wants to re-setup the root servers.

Are these strong physical machines that can be transformed into an OpenStack + OpenShift cluster or do you want to keep the current systems (+ data!) and evolve a new system?

I’m here for help, practical work or guidance. But please not that I’m also an autodidact that can strongly fail.

The most interesting question for me is how you see co-munity inside this network of providers and services. Because I have the feeling that co-munity + Ecobytes is the same for you. For me it is not. Also allmende.io remains a research prototype that could slowly evolve, if MMM would have any resources left over or a big sponsorship.

If you want to have me working on this architecture and applications, please let’s go into in depth review of your concept, if there is any, and compare it with the one I’ve clicked together. Then we can say where we work together and where each one follows an own approach.

Like ownCloud, for example. I love to have multiple apps doing different things. ownCloud combines so many different ideas into one product, that it’s hard to say if it’s good or bad. Despite the fact that there are already real large scale deploys on top of this. Let’s take the calendar functionality: Seafile doesn’t have it, ownCloud does. But the Radicale server mentionned in the Seafile quote does have LDAP integration, so does Baikal. Seafile, too. It would be easy to connect CardDavMate and CalDavZap to an existing ecosystem. In a next step I could even imagine adding OpenID functionality to these lightweight web clients.

It always depends on what you want to do.

9

Thanks @almereyda for your detailed and comprehensive post. Somehow I don’t seem to be getting Trello notifications for this board (or I’ve overseen them - I confess I get easily lost with Trello threads). I have a troublesome week with a lot of paper reviews, ill daughter and funding applications, so I’ll try to reply shortly to provide some more disambiguation and bring others in.

Ecobytes, co-munity, Degrowth conference

The first disambiguation that I think needs to be made. All the three are different things, although the projects have aspects in common. Ecobytes is a tech collective that has been running since 2004 to support collectives, artists, etc… It has been trying to survive in the last few years, this year fortunately it is gaining new power after some new people have joined and the association was formed. We do both (paid and non-paid) hosting and (usually paid) web development. co-munity is simply a Drupal/Open Atrium platform that mainly I developed/am developing. It emerged with the Beyond Our Backyards project, but quickly attracted several groups to use it, which made me understand there was a potential usefulness in further developing this. The degrowth conference is a big event taking place in September in Leipzig (I know you know this, but to contextualize for others as well) and we want to setup a users area for it, with collaboration and peer review. The participants area should be launched between mid-May and June (with a beta version for testing 15 days before)

Collaborative writing

Due to the implementation time constraints for the Degrowth conference, I’m definitely headed for Etherpad. Many people are already used to it and the time-frame of collaboration is so short (it is intended for the working groups to use it during and probably after the conference to formulate proposals together), that we probably would not benefit from many additional features. Besides that, the etherpad will be framed withing What plugins did you have in mind?

Peer review

Hypothes.is looking great! Unfortunately it seems not to have been released yet, which makes it difficult to get it on time for the conference. I did not find much about it, but since it’s built over Annotator, we can probably for now implement Annotator and hope an upgrade path is possible? (if not, we just discontinue it after the conference, no big issue I think). An important note on this: the peer review of the scientific papers will already have been passed. This will only have to do with the “stirring papers”, although it can also be used for an unofficial peer review of already accepted scientific papers. I tried to push open peer review to all the conference, but the hardcore academics are a bit difficult to convince, even in such a conference…

SSO/LDAP

Open Atrium 2 supports LDAP, including spaces and group control (https://drupal.org/project/oa_ldap), which is one of the main reasons I am very excited about using it for access control to the multiple applications. I already talked with biko about this, he can probably comment better (I just invited him). One urgent thing we will need is to import all users that register on the degrowth conference to the OA platform users area. The registration tool outputs a csv table with all user information (inc. plain text password!), so it is a matter of us deciding where and how to import. I thought of importing everything into LDAP (inc. mapping them to the specific group/space of the conference), then OA2 can work as an OpenID provider (https://drupal.org/project/openid_provider) @pierre_ozoux can also probably provide some contribution in this field.

*aaS

I’m relatively illiterate with cloud structures. I think @zeh is the right person to comment here, since he has started to deploy the new VMs. What I can say is that we cannot scrap at once all servers and go for a fully experiment. We have a lot of sites and users hosted that require stability. But we want - and need to - progressively move our servers to a new model/setup that provides more efficiency and flexibility (we still have a physical machine without any VMs!).

In general, we need manpower for new architecture developments and for the implementation of the applications. At the beginning without good perspectives of payment (well, apart from the revolutionary love), but with a scalable offer it can hopefully generate enough income to support further work on development, investment on new machines, etc… We have a good and diverse user database and a good perspective of expansion, that we can push as soon as we start providing the new generation of services :slight_smile:

1 „Gefällt mir“
10

BTW, we have a permanent meeting room on IRC where you can always come to talk (well, not 24/7 of course, but we do chat occasionaly ;)) #ecobytes @ irc.indymedia.org

11

I will get back to the rest of questions soon, but for now:

That’s just not true. Have you had a look at https://github.com/hypothesis/h ? The latest release is one year old, but master is very up to date.

12

Great! Then lets go for it. Waiting for @zeh and @pierre_ozoux to get the new servers and VMs going, then we can start working on this. I have the hope that this will happen today, since both are meeting to work on it :wink:

@almereyda do I understand well that you want to take the setup of etherpad + hypothesis on your hand?

13

Wait, regarding the architecture : are @pierre_ozoux and @zeh going for a flexible OpenStack setup or is it going to be KVM + manual Images?

This was one of my main arguements for setting up a flexible system. Let me state it like this : I’d love to set those services up, if you in return gave me the chance to work with a OpenStack + OpenShift ecosystem.

14

I like the idea of openshift on top of Docker. But I believe OpenStack is overkilling for us right now (maybe in the future)

(And just to let you know that I’m quiet new in the boat ecobytes)

Cheers!

15

Sorry, I have been a bit away, mostly fundraising with two large projects where Ecobytes is partner. If we get them both we have over 100 k € for admin and dev of web services (CSAs and degrowth groups) in the next 3 years. Now I’m a bit more available again.

@pierre_ozoux, now that you have KVM setup (and Openshift? no idea what you have setup, btw we need documentation on our internal wiki on how you have set it up), could you provide @almereyda with 2 VMs so that he can move with the implementation of node.js apps and others we want to use? @almereyda what else would we need? We absolutely need to improve the communication channels. Maybe a meeting on IRC at some point would be good to define a common working procedure, because now we have our internal schleuder list where not all of you are, an internal wiki (ikiwiki) for documentation where I think none of you have access, this tool which most of the older Ecobytes members don’t use.

How does it look like for you a meeting on Sunday or Monday evening? (approx 22:00 CET)

16

Hi guys! Yes, we’ll meet with gualter tonight at 21h german time! See you there then?

17

Sorry for not having been with you. After Paris’ OuiShare Labs Camp + Fest and a very cold farm house the week before, I’m now scratching on a Bronchitis and trying not to loose all of my involvements.

I saw some changes on the Trello board, that indicated some movement. Despite the initial refusal of Hypothes.is I’d still be very happy to provide it, next to other lovely Node partners as Ghost and Etherpad, maybe even Wiki. @pierre_ozoux and the Ecobytes elders ( @zeh I think ), *if I’m not idling on *IRC, you can also find me on Jabber / via XMPP as y@chn.io which should be a little more reliable.

If you grant me access to your documentation(s), I’d be happy to constructively criticize those representations by demanding neccessary information for my work.

@gualter Could you still offer a 2GB/2vCore VM for a Docker-Discourse-Setup? I could provide a little donation as of max. 5€/mth., if that kind of incentive already helps. @pierre_ozoux Docker’ed-OpenShift also sounds lovely. Any success?

Currently I’m in a SPAMergency move of chn.io until the end of the week and will be available again from then on.

Peace out!

18

Me also away the whole week, was organising the GROWL course in South France and tomorrow I go back to Germany. I would like to move a lot forward next week with all this Ecobytes stuff. The 2 VMs should be no problem, 5 €/mth is probably ok, it’s actually our membership value (60 €/year) and you are anyway contributing to the ecosystem :slight_smile:

19

Perfect. Let’s finally close this thread next week; and open many more!

20

So I’m basically all set. @pierre_ozoux and @zeh, how is the process going?

I’ve greatly refreshed my knowledge on PaaS and Docker-based systems by reading a little again. There’s different possibilities:

Evaluation of Docker based FOSS PaaS

Docker

Dokku

Tsuru

Flynn + Deis

non-competitive, but notable mentions

Some further readings

So I will be basically free to run the VMs as I like, right? Or do you have any constraints?