local GUI controlling local retroshare backend

RetroShare Development discussion

local GUI controlling local retroshare backend

Postby Calvin » Sat Oct 25, 2014 2:56 pm

Hi all,

It seems a lot of effort has gone into separating out the pure backend side (retroshare-nogui) from the C++/Qt presentation application. I think this means that it should be possible to have a nice clean interface between the two, with one really solid backend doing all the work, and potentially many different GUI applications running on top of it.

I've already seen some attempts like retroflux but the interface seems to be via ssh, so maybe this is a more general, networked approach than a simple local solution. On the wiki page "Documentation:retroshare-nogui" it lists two separate communication mechanisms, RPC and SSH. But an SSH key appears to be necessary for both, is that right? The blog page about DjRS talks about the "SSH RPC" protocol, so maybe they're the same?

What I'd really like to see is a "foolproof" way of getting retroshare-nogui installed from the official retroshare site, and then a "foolproof" way of providing a local cross-platform application to access it. Preferably without entering a complicated series of commands to get things connected (it has to be possible to persuade other people to run this too and they're not all technical!). I kind of assumed that if everything's on the same PC then the programs can talk to each other, but do I need to use either RPC or SSH ? Are there any significant differences between the two if they're both on the same machine?

Thanks for any tips!
Calvin
 
Posts: 9
Joined: Sat Oct 25, 2014 2:30 pm

Re: local GUI controlling local retroshare backend

Postby cave » Tue Oct 28, 2014 1:17 pm

hi,

please have a look at following links.

[list=]
[*]http://blog.freifunk.net/2014/gsoc-retroshare-social-network-plugin-review-and-future
[*]https://github.com/cristeab/RS-QML-GUI && http://tesseract.ws/portfolio/ui
[*]https://www.reddit.com/r/retroshare/comments/2cmkj3/retroshare_webinterface_four/
[*]http://blog.freifunk.net/2014/gsoc-retroshare-social-network-plugin-compared-retroshare-forums
[*]https://wiki.cavebeat.org/images/a/a4/Retroshare_Webinterface_4_14_07_29.pdf
[*]https://wiki.cavebeat.org/images/4/4f/Retroshare_JSON_over_http_API_14_07_29.pdf
[/list]

AND

http://vps.semesterofcode.com/projects/browse

(status: open)
Webinterface for a decentralized social network

Motivation: offer similar features as facebook or diaspora. But don't expose any private data to a server operator. Retroshare is decentralized and does not need servers.

During Google Summer of Code 2014 i started a social network plugin for Retroshare (original Idea). It offers similar features as a facebook wall. This plugin works, but it turned out that the choosen front end technologie (Wt) is unflexible. With Wt the webinterface is written in C++. This is nice if you don't have a clue about web technologies (like me half a year ago). Then a webdeveloper introduced me into the world of JavaScript/HMTL/SASS and now i think a webinterface should be made directly with web technologies, not with C++.

Desired Outcome: a webinterface for a social network. Create a post, display comments, allow to make comments or recommend posts to others. A profile page with a profile picture. Later we want a user interface to group persons into circles to allow access retrictions. The task could be extended to cover all Retroshare features. These are chat, croup chat, mail, forums and channels. We could also add long wanted features like photo albums. The interface should work with touch screens.

Communication with the Retroshare core will happen with JSON encoded messages over http or Websockets. This interface is under construction and should be available until December.

Mentor: Tillmann Gansky

Programming languages used: JavaScript, HTML, CSS/SASS

This project is right for you, if you don't fear design+layout and if you like a visible outcome of the project.
Statistics
Number of proposals already submitted to this project (0)
The project mentor has not marked any proposal as their preferred solution yet.
cave
 
Posts: 109
Joined: Tue Nov 13, 2012 10:27 pm

Re: local GUI controlling local retroshare backend

Postby Calvin » Tue Oct 28, 2014 4:18 pm

Thanks for those links, cave, that looks excellent. I could well get behind a web-based frontend.

I guess you're the author of those two pdfs, but I also guess that this forum isn't where these things are being discussed. Has anything happened since July or has it gone quiet again? Retroshare itself is still on 0.5.5 as far as I can see, so does that mean that the server side of this proposed http/json interface isn't available yet?

Where can I find out more about the current status, is it all being discussed somewhere inside retroshare rather than here on the forum?

Thanks!
Calvin
 
Posts: 9
Joined: Sat Oct 25, 2014 2:30 pm

Re: local GUI controlling local retroshare backend

Postby cave » Wed Oct 29, 2014 9:20 am

no problem,

Wrong guess. I am not the author. I am just mirroing the files in the ClearWeb. All honor belongs to electron. http://blog.freifunk.net/tillmann-gansky
I am not sure if he posted it on the Freifunk-Developer-GSoC Blog or if he posted it inside RetroShare in the Developer Forums.

RetroShare stable latest release is v0.5.5c r7589


There is a Branch for rssocialnet in v06. but this branch was already merged into v06/trunk http://sourceforge.net/p/retroshare/code/7646/tree/branches/v0.6-rssocialnet/

http://sourceforge.net/p/retroshare/code/7492/
Merge of branch v0.6-rssocialnet 7419 to 7488. Changes from electron and myself:

- added possibility to modify groups (e.g. edit circles)
- fixed mismatched free/delete in fimonitor.cc, authssl.cc, pqibin.cc (saving encrypted hash cache file)
- improved plugin interface class to allow plugins to access GXS objects.
- added method to un-register notify clients from RsNotify
- fixed pqisslproxy for windows, due to win not properly supporting sockets in non blocking mode.
- removed static members form RsInitConfig and made RsAccounts object a pointer. This prevents plugin initialisation problems at symbol resolving time.
- removed bool return from p3IdService::getOwnIds()

I think only GXS Backend Stuff got into trunk with this merge.

I do not know about state for the http/json interface. I hope the development continues and we can see some results soon. electron wants to focus on the Backend C++ coding and is looking for someone with better Web/design skills for the frontend to work together.

These both Interfaces have been written as a Plugin with Wt http://www.webtoolkit.eu/wt
https://gitorious.org/rswebui && https://gitorious.org/rssocialnet


If you want to get in contact, a good idea would be to use RetroShare :mrgreen:
this forum is in a Zombie state, and for example i do not look regularly into this place because the Communication is mostly done inside RetroShare and the Developer Forums.

v06 is open Beta and you can follow the development http://sourceforge.net/p/retroshare/code/HEAD or via Feed http://sourceforge.net/p/retroshare/code/feed

lots of alpha/beta stuff and projects is available here https://github.com/retroshare

If you are using v05.5 you can send me your key and we can talk directly. additionally build v06 yourself and connect on the v06 network. Most stuff happens there at the moment.

best regards
cave
cave
 
Posts: 109
Joined: Tue Nov 13, 2012 10:27 pm

Re: local GUI controlling local retroshare backend

Postby Calvin » Wed Oct 29, 2014 9:19 pm

cave wrote:Wrong guess. I am not the author. I am just mirroing the files in the ClearWeb.
Ok, sorry for the confusion. And thanks for taking the time to publish this stuff on the web also. I can't understand why someone would write that they need to attract developers, and then publish those comments where those future developers can't see it.

I do not know about state for the http/json interface. I hope the development continues and we can see some results soon. electron wants to focus on the Backend C++ coding and is looking for someone with better Web/design skills for the frontend to work together.
That sounds very promising. I look forward to a stable release of the backend at least.

If you are using v05.5 you can send me your key and we can talk directly. additionally build v06 yourself and connect on the v06 network. Most stuff happens there at the moment.
Eek, you make it sound like some things are discussed on the 0.5.5 network and other things are discussed on the 0.6 network. I hope I misunderstood that. For sure we can discuss it in private if you like, but personally I don't feel the need for secrecy, and the "official" retroshare development forum here seems a reasonable place for other people to look for information on this topic, don't you think?

I have a few very simple, high-level questions, if you don't mind. Or if anyone else knows too, of course!

The proposed web frontend, is this intended to run on the same physical machine as the retroshare node, or not? It seems odd to encrypt traffic all the way to the retroshare node, and then transmit it across the network with http. Would https not be better?

If I understand correctly, the proposal is for retroshare-nogui (the purely backend part) to have a built-in web server offering the json. But because this purely backend part doesn't know anything about the gui, this web server can't serve up the actual web pages for the gui, right? So if the web frontend runs in a browser (which I think is the idea), then what address will this point to - would this be a second web server which will serve up the html, js, css, images etc (also unencrypted)? And the javascript inside the browser would then request the json from the "nogui" webserver?

If I understand correctly, the current GUI talks to the no-gui backend directly because they're both C++ and compiled and linked together - that's why it doesn't need the whole ssh-rpc thing but with the disadvantage that it's all got to be tightly integrated (and compiled for each platform). But just because we want a clean interface, does that mean the best solution is having a webserver and http with GETs and POSTs? Isn't there a way to make the current ssh-rpc interface more user-friendly or easier to set up, and then any web gui could use that interface to talk to the backend, while still sending html/css/js to the client browser from its web server. Which sounds like retroflux. But maybe other native GUI applications would become more viable too.

One thing not mentioned at all in those pdfs is the goal of making the gui modular or at least highly customisable. Different people have strong opinions about what should be in the gui and what not, so there need to be ways for each user to remove features they don't want. But you said the focus is on the backend interface for now, which sounds perfect.
Calvin
 
Posts: 9
Joined: Sat Oct 25, 2014 2:30 pm

Re: local GUI controlling local retroshare backend

Postby electron » Sun Nov 02, 2014 7:52 pm

Calvin wrote:It seems a lot of effort has gone into separating out the pure backend side (retroshare-nogui) from the C++/Qt presentation application. I think this means that it should be possible to have a nice clean interface between the two, with one really solid backend doing all the work, and potentially many different GUI applications running on top of it.

Retroshare has a backend (libretroshare) and a GUI. Libretroshare has a clean interface, but it is C++ only. So it would be totally possible to have different GUI applications, as long as they are written in C++.

Calvin wrote:On the wiki page "Documentation:retroshare-nogui" it lists two separate communication mechanisms, RPC and SSH. But an SSH key appears to be necessary for both, is that right? The blog page about DjRS talks about the "SSH RPC" protocol, so maybe they're the same?

retroshare-nogui has an interface based on ssh. ssh can be used to transport Retroshare specific RPC packets, this is called ssh-rpc. ssh can also be used to deliver a text based interface to a ssh client like PuTTY. But the menu interface does not have any usefull functions, it was only made for testing.

Calvin wrote:What I'd really like to see is a "foolproof" way of getting retroshare-nogui installed from the official retroshare site, and then a "foolproof" way of providing a local cross-platform application to access it. Preferably without entering a complicated series of commands to get things connected (it has to be possible to persuade other people to run this too and they're not all technical!).

I agree. If it is to complicated to use no uses it, it is useless. Even people like me, who know how it works, don't use it because it is to complicated. The Problem is the network, it requires a reliable name resolution system, an accessible port and encryption.
In the future i want to run a full Retrposhare node on every system. But i still want a separation into core and user interface. And i want to make a user interface based on webtechnologies like HTML/JavaScript/CSS. I think these technologies are flexible, portable and they offer unlimited possibilities. The interface between gui and core will be prepared for use over network, but i don't think it will be the main use case.

Calvin wrote:Has anything happened since July or has it gone quiet again? Retroshare itself is still on 0.5.5 as far as I can see, so does that mean that the server side of this proposed http/json interface isn't available yet?

Time passed faster than expected ;). I explored different frameworks like Angular, React, Bootstrap und the tools around them like the node js. After thinking about this for a while i finally have a plan for most of the problems. The current status of the code is not more than what you can see in the gitorious repository.

Calvin wrote:Where can I find out more about the current status, is it all being discussed somewhere inside retroshare rather than here on the forum?

I had some discussions in private chat where i received very useful input. Everything else is being dicussed in my head. If you are interested in technical details i would be pleased to share them.

Calvin wrote:Eek, you make it sound like some things are discussed on the 0.5.5 network and other things are discussed on the 0.6 network.

v0.6 has a much better forum system. And we need to test it. So v0.6 issues are mainly posted in v0.6 forums.

Calvin wrote:The proposed web frontend, is this intended to run on the same physical machine as the retroshare node, or not? It seems odd to encrypt traffic all the way to the retroshare node, and then transmit it across the network with http. Would https not be better?

Yes, i want to run it on the same system, and even access it from the same system. https is not secure, because there is no good way to check the certificate. Still, if someone wants to add https, it will be possible.

Calvin wrote:If I understand correctly, the proposal is for retroshare-nogui (the purely backend part) to have a built-in web server offering the json. But because this purely backend part doesn't know anything about the gui, this web server can't serve up the actual web pages for the gui, right?

All the GUI state will be handled in JavaScript on the client side. So the server will serve static html and JavaScript.

Calvin wrote:So if the web frontend runs in a browser (which I think is the idea)

It can run in a Browser and get the data over http. But this will not be required. We can also bundle the static files with a native launcher application. First i will develop using a browser(for easier debugging), and then we will see what to do next.

Calvin wrote:If I understand correctly, the current GUI talks to the no-gui backend directly because they're both C++ and compiled and linked together - that's why it doesn't need the whole ssh-rpc thing but with the disadvantage that it's all got to be tightly integrated (and compiled for each platform). But just because we want a clean interface, does that mean the best solution is having a webserver and http with GETs and POSTs?

Yes, current intefrace between GUI and libretroshare is C++ only. Yes, http is not required.

Calvin wrote:Different people have strong opinions about what should be in the gui and what not, so there need to be ways for each user to remove features they don't want.

Yes, there are different opinions. But currently this is not and issue, because there are not enough developers to maintain different user interfaces. With the new interface it will be easier to modify the user interface, because you don't need a compiler and all the complicated setup. And you can see your modifications immediately, you don't have to wait for compiler, linker, and startup. This immediate-feedback thing with webtechnologies is something i like very much.
electron
 
Posts: 96
Joined: Sun Aug 12, 2012 9:39 am

Re: local GUI controlling local retroshare backend

Postby cave » Mon Nov 03, 2014 11:36 am

cave wrote:Eek, you make it sound like some things are discussed on the 0.5.5 network and other things are discussed on the 0.6 network. I hope I misunderstood that. For sure we can discuss it in private if you like, but personally I don't feel the need for secrecy, and the "official" retroshare development forum here seems a reasonable place for other people to look for information on this topic, don't you think?

erm, yes, there is some stuff discussed in v05 and in v06 networks. Most developers and techies and beta testers are running vo5 and v06 clients. In v05 all the folks and the warez stuff and random forums are in use.
In v06 just the technical retroshare forums are populated. The Forum system and the GXS system just works better than in v05. If i find a bug, or want any specific topic discussed with the developers, i just post on the developer forum.
The transition Phase has just begun and early adopters are moving from v05 to v06, and v05 starts to fade out (just the start).

Regarding secrecy concerns, its not illegal to discuss here and i do not have a problem with the web forum, but the RetroShare forums are more convenient. And the respect your privacy on a better scale.
And its a good idea to go dark because of OSINT - > https://en.wikipedia.org/wiki/Open-source_intelligence There is no need to offer them all informations.

regarding development its a drawback. possible developers just don't have a chance to know what is going on inside RetroShare, because they are not able to see it without access to the forums.


calvin wrote:I have a few very simple, high-level questions, if you don't mind. Or if anyone else knows too, of course!

Ask as much as you can. We will try to answer, at least i'll try to find someone to answer.


calvin wrote:The proposed web frontend, is this intended to run on the same physical machine as the retroshare node, or not? It seems odd to encrypt traffic all the way to the retroshare node, and then transmit it across the network with http. Would https not be better?

If the WebServer binds to localhost (the node/core is on the same machine) i think http is enough. encryting it by design would be better of course.
In a next step its a good idea to make it available via https, and afterwards to make it available through the internet. But like electron said there are other problems like check of the certificates for secure https, port forwardings (this maybe solves with IPv6 in the next century), dns name resolving.



calvin wrote:If I understand correctly, the proposal is for retroshare-nogui (the purely backend part) to have a built-in web server offering the json. But because this purely backend part doesn't know anything about the gui, this web server can't serve up the actual web pages for the gui, right? So if the web frontend runs in a browser (which I think is the idea), then what address will this point to - would this be a second web server which will serve up the html, js, css, images etc (also unencrypted)? And the javascript inside the browser would then request the json from the "nogui" webserver?

I am not sure if the WebServer could be an extension/plugin. the plugin could be available for gui and non-gui.

the other parts were answered by electron
cave
 
Posts: 109
Joined: Tue Nov 13, 2012 10:27 pm

Re: local GUI controlling local retroshare backend

Postby Calvin » Fri Nov 07, 2014 5:58 pm

Thanks a lot to both cave and electron for answering my questions, I really appreciate it. And yes I understand your preference for discussing these things in a more restricted way. So thanks :)

About the web server, yeah what you say makes a lot of sense and I'm not sure why I thought you'd need two webservers. Of course the nogui part can have the webserver and the webgui part can just provide the static resources for the nogui webserver to deliver.

Just one more question - is your intention to remove the current ssh-rpc interface, or will the new json-http interface be an additional way of controlling the backend? I assume the existing guis still need to be supported. Is there likely to be a choice whether to run the backend with just ssh-rpc, or just json-http, or neither, or both?

I think to run 0.6 I need to compile from source, right? Although you talk about gitorious, it seems like the right one to take is from sourceforge's svn, right?
Calvin
 
Posts: 9
Joined: Sat Oct 25, 2014 2:30 pm

Re: local GUI controlling local retroshare backend

Postby electron » Sat Nov 08, 2014 9:03 am

Calvin wrote:Just one more question - is your intention to remove the current ssh-rpc interface, or will the new json-http interface be an additional way of controlling the backend? I assume the existing guis still need to be supported. Is there likely to be a choice whether to run the backend with just ssh-rpc, or just json-http, or neither, or both?

The current ssh-rpc and the new rpc interface can live besides each other. There are still a few issues, for example chat messages will only be delivered to one interface (This is also why ssh-rpc is only available for nogui). But these issues are solveable.

Calvin wrote:I think to run 0.6 I need to compile from source, right? Although you talk about gitorious, it seems like the right one to take is from sourceforge's svn, right?

No, you don't have to compile yourself. Yes, the sources live in the sourcefoge svn. For Ubuntu there is the ppa like usual: https://launchpad.net/~csoler-users, Windows builds are distributed through a Retroshare channel.
electron
 
Posts: 96
Joined: Sun Aug 12, 2012 9:39 am

Re: local GUI controlling local retroshare backend

Postby javars » Sun Nov 30, 2014 2:27 am

The REST API would be invaluable for extending the front-ends; not just for web, but for mobile, other languages like Java, etc,...

I've been developing since 1979 and might like to contribute to this project. While I did plenty of C and C++ in the 80s and 90s, I really love the productivity of Java today. Java could not only allow a rich web UI if a good REST API is available, but also a good Android app, both of which would only need ports 80/443, extending the reach greatly.

As for SSL, while using secure client certs is certainly possible, I agree it isn't user-friendly and I never liked how it was tied to the browser instead of the user. Nevertheless, it shouldn't take much for those who want to use it to setup their servers to only use client certs signed by a trusted CA. This would not be the common use case for the average Joe, though.

Better would be security in the REST API, or via a security layer on top of the BASE API. You could argue that if your Java App server is on the same secured host as your RetroShare, it would not need a very secure interface. Only the API exposed to the browser would need that.

'Course, I don't know enough about the current RS APIs to know if this wouldn't be automatic. But, if it's relying on SSH for security, and REST wouldn't have that, then securing the REST would have to be on TODO.

I look forward to getting 0.6 and joining the forums there. Right now, waiting for moderator to release my build issue so someone can hopefuly help me complete the build. Most of my C days was prior to Linux being born, and primarily on CP/M, DOS and Windows, so I've never had to create a MAKE on Linux -- not that I'd ever want to return to DLL hell. :)
javars
 
Posts: 4
Joined: Sun Nov 30, 2014 1:18 am

Next

Return to Developers Corner

Who is online

Users browsing this forum: No registered users and 1 guest

cron