Back

Peerfetch – Peer-to-Peer HTTP over WebRTC

280 points3 monthsgithub.com
tarasglek3 months ago

Doing http over webrtc is how https://camect.com works to let one access cameras own private server via their ui. They have a centralized bit for auth and then use webrtc and a physical nvr to serve your videos maximally efficiently...so there is low risk of their cloud becoming a financial burden that they cancel ala google nest cams

It's a super nice architecture

kabes3 months ago

So from your explanation I get that they use webrtc for videos. But then what do they use http over webrtc for? Do they serve the UI as well over webrtc?

tarasglek3 months ago

Their ui is also hosted on the nvr, they serve ui assets over webrtc

londons_explore3 months ago

> have a centralized bit for auth

Theoretically they could get the camera itself do auth, and then the server becomes fully 'dumb', and need not even be service specific.

tehlike3 months ago

Oauth is nice and convenient.

Saris3 months ago

Sounds kind of similar to Frigate but packaged up ready to go, neat.

tehlike3 months ago

I am an early backer and have been using it for few years bow. Camect is great.

cyanydeez3 months ago

How soon do you think IT will clamp down on webrtc?

robertclaus3 months ago

Clever idea. Begs the question of why you would use http if you already have a bidirectional webRTC connection, but I guess it depends on the application.

ongy3 months ago

Provide a server on-prem at the customer, but allow them a hybrid access to the system.

Via cloud when necessary, "local" (by WebRTC) when possible. While we could just open a local port, using the cloud to arbitrate gives us a common product vision, and proper authN/authZ.

Also allows us to pull the latency down to single digit milliseconds. The regional relays are double digit. When we use relays that aren't regional it's a couple hundred.

throwawaymaths3 months ago

You can securely serve a webpage from a server behind NAT without creating a VPN and without https certificated

dlenski3 months ago

I suppose the persistence of IPv4 has broken all of our brains, but with IPv6 you can just Not Have NAT, and just have normal end-to-end connectivity to any random box in your home from outside.

(And yes, I do this. Works great.)

klabb33 months ago

IPv6 can get rid of NAT which is one of the most annoying hurdles. It unlocks the type of use case where technical people can host something from home for fun, although many can’t access it because both parties need ipv6.

But if you set your sights higher and want to build true p2p apps for non-techies, or if you want “roaming” servers (say an FTP server on your laptop), there are more obstacles than NAT, in practice:

- Opening up ports in both a residential router and sometimes the OS or 3p firewall. Most people don’t know what a port is.

- DNS & certs which require a domain name and a fixed connection (if the peer moves around across networks, eg a laptop or phone, DNS is not responsive enough)

dlenski3 months ago

> IPv6 can get rid of NAT which is one of the most annoying hurdles.

Right.

> It unlocks the type of use case where technical people can host something from home for fun, although many can’t access it because both parties need ipv6.

At this point, that's an obstacle, but at some future point hopefully IPv6 will hit a critical mass and network effects will take off because there'll be enough stuff that _doesn't work without IPv6_, so customers will demand it.

> if you set your sights higher and want to build true p2p apps for non-techies

Definitely.

Restoring the universal endpoint-to-endpoint connectivity on the IP network overcomes a _major hurdle_, a hurdle that's so big and longstanding that people have come to just assume its existence and fear and removal… but it certainly doesn't solve all the problems.

> or if you want “roaming” servers (say an FTP server on your laptop)

https://en.wikipedia.org/wiki/Multipath_TCP will take a big dent out of this, I think.

> - Opening up ports in both a residential router and sometimes the OS or 3p firewall. Most people don’t know what a port is.

I mean, UPnP makes big improvements in this area, but a lot of devices stupidly don't handle it, or block it for alleged security reasons. Frustrating.

> - DNS & certs which require a domain name and a fixed connection (if the peer moves around across networks, eg a laptop or phone, DNS is not responsive enough)

There's no real reason why TLS clients _must_ only trust certs when they see that the CN or SAN matches the _domain name_ through which they looked up the IP address. I think that with a better issuing infrastructure and UX, a TOFU-based (https://en.wikipedia.org/wiki/Trust_on_first_use) approach to self-signed certs for peer-to-peer services could be both comprehensible for non-techies and highly secure.

concrete_head3 months ago

Would you be willing to share a few details on how you do this. And how do you prevent someone spamming your devices or is the risk so low you don't care?

Unfortunately most ISP's in my area don't dish out IPv6 addresses without ridiculous monthly charges. I hope one day it becomes more commonplace.

jeroenhd3 months ago

> Unfortunately most ISP's in my area don't dish out IPv6 addresses without ridiculous monthly charges

If you've got an IPv4 address that responds to ICMP, HE's https://tunnelbroker.net/ offers free IPv6 ranges (a bunch of /64s and a /48) for free. You can configure a tunnel to work through many routers, but with some setup you could also have something like a Raspberry Pi announce itself as an IPv6 router.

Sites like Netflix treat HE tunnels as VPNs, though, so if you run into weird playback errors, consider configuring your device's DNS server/network not to use IPv6 for that.

As for your questions:

> how you do this

Open port 8888 to (prefix):abcd:ef01:2345:56, or whatever IP your device obtains, in your firewall. It's the same process as with IPv4, except you can use the same port on multiple devices.

> And how do you prevent someone spamming your devices or is the risk so low you don't care?

While some services have started scanning IPv6, a home network from a semi-competent ISP will contain _at least_ 2^64 IPv6 addresses. Scanning the entire IPv6 network is unfeasible for most automated scanners.

ffsm83 months ago

> And how do you prevent someone spamming your devices or is the risk so low you don't care?

That's the job of a firewall and is unchanged between ipv4 and IPv6. Theyre both equally vulnerable to denial of service attacks

+2
immibis3 months ago
justsomehnguy3 months ago

They need to find them first.

throwawaymaths3 months ago

There's something nice about being anonymous behind a communal v4 gateway.

Also can you get an tls cert for a ipv6 number address? Or are you punching through using only ssh or unencrypted stuff?

+3
dgl3 months ago
fulafel3 months ago

Though WebRTC works great with IPv6 too. Then the use case would be running it on a server that has incoming connections firewalled.

hcfman3 months ago

There’s lots of ipv6 available. But it’s not everywhere yet

ycombinatrix3 months ago

This supports UDP/unreliable data streams in the browser tho.

dingi3 months ago

I don't think that's possible without a jump server. If all peers are NATed, there is no way doing p2p without a jump server. WebRTC is a giant rabbit hole itself.

evbogue3 months ago

This idea makes me want cjdns and/or yggdrasil over websockets.

immibis3 months ago

Don't you want static peering setup for them?

meiraleal3 months ago

It is really annoying when someone posts an interesting project and HN has a big discussion but when I get to try to lib, it is unmaintained and the last update was 3 years ago.

There were great recommendations in this thread tho, thanks a lot! This one looks good: https://github.com/subins2000/p2pt

evbogue3 months ago

If we're talking about great p2p WebRTC libraries, try Trystero: https://github.com/dmotz/trystero

meiraleal3 months ago

Wow, really impressive project! Thanks.

whitehexagon3 months ago

Is there a way to do this without the signaling server?

iod3 months ago

Webtorrent

Use a free signalling server from the webtorrent community. You can skip the torrent part of the implementation and just use the signalling, it's awesome. You can use a libraries like:

https://github.com/webtorrent/bittorrent-tracker

https://github.com/subins2000/p2pt

to get started. For me, I found the protocol is simple enough where I just use small vanilla javascipt implementation to talk to the websocket servers to generate the signalling messages. I wish more people knew about this and realize how easy it can be to bring WebRTC to their applications.

List of some free webtorrent trackers:

  wss://tracker.openwebtorrent.com
  wss://tracker.files.fm:7073
  wss://tracker.webtorrent.dev
---> Usage stats for the last one: https://tracker.webtorrent.dev

Some free stun servers for NAT traversal:

  stun:stun.cloudflare.com
  stun:stun.l.google.com:19302
nhkcode3 months ago

This is super cool and almost makes it possible building PWAs that only need a dumb http server to deliver the app as a bunch of static files and still allow users to synchronize data between their devices. It still depends on the tracker but if the user could change the tracker it sounds like it's currently the best way to get clients to communicate with each other without depending on a server provided by the PWA.

catapart3 months ago

Thank you for this! I knew this shit was done by someone already and I've spent two years resisting the urge to re-invent this wheel. p2pt is exactly what I knew was possible and have been looking for!

turblety3 months ago

I think the closest you might get is something like the bittorrent dht. There are still bootstrap servers for the first few connections, but there's really no getting away from that, right?

https://github.com/webtorrent/bittorrent-dht

tfolbrecht3 months ago

Not with the three major browsers and NAT unfortunately.

https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/...

wuiheerfoj3 months ago

check out NAT hole-punching in libp2p: https://docs.libp2p.io/concepts/nat/hole-punching/

scroll down a bit for the STUNless/TURNless bit

dmotz3 months ago

You might be interested in my side project Trystero https://github.com/dmotz/trystero

It abstracts away the work of signaling and connects peers via open decentralized networks like BitTorrent, Nostr, MQTT, IPFS, etc.

fullspectrumdev3 months ago

Not had time to read through the docs properly, but would this work for CLI apps (Node compiled to an executable?) or similar?

The problems this solves look interesting for a few CLI tools I want to build :)

evbogue3 months ago

I am also curious about the progress on this feature: https://github.com/dmotz/trystero/issues/24 for more info

Sean-Der3 months ago

Yep! If you do one non-browser WebRTC agents https://github.com/pion/offline-browser-communication

This could be possible browser to browser! I don’t think that has ever been an official use case

thoronton3 months ago

I don't get it. Where is the signaling server and how is it working?

thrdbndndn3 months ago

(Aside) speaking of WebRTC, but is there any solution to record videos that is done by webRTC?

There are already more than enough tools that can record HLS and Dash, but I haven't find anything, not even PoC that can record video streams transited via WebRTC (e.g. agora.io).

HyprMusic3 months ago

https://recordrtc.org/ should be able to do it.

Sean-Der3 months ago

Do you want this in the browser or a specific language?

chfritz3 months ago

Using webrtc is actually overkill in this case. What you really want is just the ICE part used by webrtc to establish a peer-to-peer connection. That's what tailscale does, too. Webrtc then adds a lot of media-related protocols on top of that connection, which, of course, the Peerfetch application doesn't care about.

jiehong3 months ago

Nice!

Although, an alternative is something like Tailscale.

etherdream3 months ago

I have tried this idea before, combining Service Worker to implement a decentralized website.

localfirst3 months ago

got excited but the repo hasn't been updated for over 3 years.

lerpgame3 months ago

its advertising that it’s secure e2e even behind firewall/etc but that’s not true because webrtc will fallback to using TURN server to relay when other methods fail which will break the encryption, just fyi.

WatchDog3 months ago

WebRTC won’t use TURN unless it’s explicitly configured with a TURN server. Even if it did use a TURN server webrtc is still e2e encrypted.

You need to trust the signalling server though.

This library seems to do a few other things, which maybe reduces the trust in the signalling server, but I didn’t really read it in enough detail to comment on it.

Sean-Der3 months ago

Connection is E2E encrypted when using TURN. Using TURN has no negative impact on security.

The TURN server can see the size/src/dst so that has a privacy implication!

strbean3 months ago

You can pass configuration to disable ICE entirely.

Looks like it's using PeerJS, which defaults to a config of using a Google STUN server and no TURN servers. Not sure if using a STUN server compromises the E2E in some way?

server_man30003 months ago

Why would STUN compromise e2e? STUN just returns your IP

strbean3 months ago

I just didn't want to speculate, as I'm not familiar with the security considerations here.

But, thinking about it a bit, couldn't a compromised STUN server establish a MITM by lying to you about your IP, and then relaying to you? This old HN comment describes it: https://news.ycombinator.com/item?id=11192610

I don't know if this would break the E2EE here (although if it wouldn't, I'm not sure how a TURN server would either, as that's just a baked in MITM).

lerpgame3 months ago

i was wrong actually, it doesn’t weaken security as long as the data is encrypted either using DLTS or application layer encryption, please ignore my comment lol.