How-to guides on EdgeVPN

Run as a VPN

Mon, 01 Jan 0001 00:00:00 +0000

To start the VPN, simply run edgevpn without any argument.

An example of running edgevpn on multiple hosts:

# on Node A
$ EDGEVPNTOKEN=.. edgevpn --address 10.1.0.11/24
# on Node B
$ EDGEVPNTOKEN=.. edgevpn --address 10.1.0.12/24
# on Node C ...
$ EDGEVPNTOKEN=.. edgevpn --address 10.1.0.13/24
...

… and that’s it! the --address is a virtual unique IP for each node, and it is actually the ip where the node will be reachable to from the vpn. You can assign IPs freely to the nodes of the network, while you can override the default edgevpn0 interface with IFACE (or --interface)

Addressing and DHCP

Mon, 01 Jan 0001 00:00:00 +0000

Every VPN node needs a virtual address on the edgevpn0 interface. There are three ways to get one, and two extra flags that change how packets are routed once you have it.

Static addresses with `--address`

--address takes a CIDR and defaults to 10.1.0.1/24. It is the address the node is reachable at from inside the VPN, and it must be unique across the network — nothing checks this for you.

IPv6

Mon, 01 Jan 0001 00:00:00 +0000

Experimental feature. IPv6 support is provisional and has known gaps — see below before relying on it.

IPv6 works with static addresses only. One address per interface; dual stack is not supported, so a node is either IPv4 or IPv6, not both.

$ EDGEVPNTOKEN=.. edgevpn --address fd:ed4e::11/64 --mtu 1500

Two things to get right:

The address must be static. --dhcp allocates IPv4 addresses only, so it cannot be combined with an IPv6 --address.
--mtu must be above 1280, the IPv6 minimum link MTU. EdgeVPN’s default is 1200, which is below it, so you have to set --mtu explicitly.

Tracking issue #15 is still open at the time of writing; it is the place to check for the current state of IPv6 support.

Enable the DNS server

Mon, 01 Jan 0001 00:00:00 +0000

Experimental feature!

DNS Server

A DNS Server is available but disabled by default.

The DNS server will resolve DNS queries using the blockchain as a record and will forward unknown domains by default.

It can be enabled by specifying a listening address with --dns. For example, to bind to default 53 port locally, run in the console:

edgevpn --dns "127.0.0.1:53"

To turn off dns forwarding, specify --dns-forwarder=false. Optionally a list of DNS servers can be specified multiple times with --dns-forward-server.

Send and receive files

Mon, 01 Jan 0001 00:00:00 +0000

Sending and receiving files

EdgeVPN can be used to send and receive files between hosts via p2p with the file-send and file-receive subcommand.

Sending and receiving files, as services, don’t establish a VPN connection.

Sending

$ edgevpn file-send --name unique-id --path /src/path

Receiving

$ edgevpn file-receive --name unique-id --path /dst/path

Tunnel TCP services

Mon, 01 Jan 0001 00:00:00 +0000

If you have not done this before, the step-by-step version is Share a service between two hosts.

Forwarding a local connection

EdgeVPN can also be used to expose local(or remote) services without establishing a VPN and allocating a local tun/tap device, similarly to ngrok.

Exposing a service

If you are used to how Local SSH forwarding works (e.g. ssh -L 9090:something:remote <my_node>), EdgeVPN takes a similar approach.

A Service is a generalized TCP service running in a host (also outside the network). For example, let’s say that we want to expose a SSH server inside a LAN.

HTTP egress and the proxy

Mon, 01 Jan 0001 00:00:00 +0000

Only plain HTTP is proxied. HTTPS does not work — see HTTPS does not work below.

EdgeVPN can designate one or more nodes as HTTP egress nodes. Another peer runs edgevpn proxy, which exposes an ordinary local HTTP proxy. Requests sent to that proxy travel over libp2p to one of the egress nodes, which performs the request from its own network and streams the response back.

What this is, and what it is not

This proxies HTTP requests, not arbitrary IP traffic. It is not a VPN exit node: nothing is rerouted at the IP layer, your default route is untouched, and only the clients you explicitly point at the local proxy are affected. If you want a real network interface between peers, that is run as a VPN instead.

Relays and hop nodes

Mon, 01 Jan 0001 00:00:00 +0000

Two nodes behind NAT cannot always dial each other. EdgeVPN tries hole punching first (--holepunch, on by default), but hole punching needs a third party that both peers can already reach, and it does not work through every NAT. A relay is that third party: a node on a reachable address that other peers connect through.

A relay does not need a VPN interface, an IP on the virtual network, or root. edgevpn start is the command for it:

Ledger ownership

Mon, 01 Jan 0001 00:00:00 +0000

--ownership is the one EdgeVPN setting that every node on a network must agree on. Mixing off with enforce produces a network that looks up but silently drops half its ledger. If you are upgrading an existing network, read Changing the mode on a live network first.

Every peer that holds the network token can write to the ledger. Without ownership, it can write to anyone’s entries: overwrite the machines record that maps an IP to a peer, claim someone else’s DNS name, or replace a service announcement. Ownership closes that gap by binding each entry to the libp2p identity that wrote it.

Trusted networks

Mon, 01 Jan 0001 00:00:00 +0000

Experimental feature!

Trust zones are the only mechanism in EdgeVPN that can exclude a peer holding a valid network token at runtime — everything else, the token itself, ledger ownership and relay ACLs, assumes every token holder is a full member. If your membership is a fixed set of hosts, --static-peertable excludes token holders too, without being experimental; see static peer tables.

Read the security model first either way. It covers what peergating adds, what it deliberately does not add, and the admission gap in the current implementation that you should weigh before relying on it.

Use EdgeVPN as a library

Mon, 01 Jan 0001 00:00:00 +0000

EdgeVPN can be used as a library. It is very portable and offers a functional interface.

To join a node in a network from a token, without starting the vpn:

import (
    "github.com/ipfs/go-log"
    "github.com/mudler/edgevpn/pkg/discovery"
    "github.com/mudler/edgevpn/pkg/logger"
    node "github.com/mudler/edgevpn/pkg/node"
)

d := discovery.NewDHT()
m := &discovery.MDNS{}

e, err := node.New(
    node.Logger(logger.New(log.LevelInfo)),
    node.MaxMessageSize(2 << 20),
    node.FromBase64(mDNSEnabled, DHTEnabled, token, d, m),
    // ....
)
if err != nil {
    return err
}

if err := e.Start(ctx); err != nil {
    return err
}

node.FromBase64 decodes the same base64 token the CLI uses (edgevpn -g -b) and wires the discovery services into the node, which is why it takes the *discovery.DHT and *discovery.MDNS values you built above. The two booleans enable mDNS and DHT discovery respectively.

Run with Docker

Mon, 01 Jan 0001 00:00:00 +0000

The repository ships a docker-compose.yml that brings up a single VPN node. It is short, and every line in it is there for a reason. This page explains those reasons, so you can adapt it instead of copying it.

For the published image and its tags — in particular why :latest is a development build and not the newest release — see install. Everything below assumes you have picked a tag.

Running the VPN in a container gives it the same reach as running it on the host: with network_mode: host and NET_ADMIN it creates a systemwide network interface and can reconfigure host networking. The container boundary is not buying you isolation here.

Persist node identity and state

Mon, 01 Jan 0001 00:00:00 +0000

This page has not been written. By default an EdgeVPN node generates a fresh libp2p key on every start, so its peer ID changes on every restart. Under --ownership enforce (the default) that orphans everything the node had announced. The flags that change this are documented only by their one-line --help text.

What is missing, and where the source is:

--privkey-cache (env EDGEVPNPRIVKEYCACHE, off by default, marked experimental) and --privkey-cache-dir (env EDGEVPNPRIVKEYCACHEDIR, defaulting to $HOME/.edgevpn). The implementation is in cmd/util.go: it reads or generates <dir>/privkey, writing the directory 0700 and the file 0600. The comment above it explains why it is not enabled automatically — the default directory is per-user, so two co-located EdgeVPN processes sharing it would boot with the same peer ID. Each node needs its own directory.
The interaction with ownership. cmd/util.go emits a warning when ownership enforcement is on and the identity is ephemeral, because the node’s entries are reclaimed after the liveness TTL on every restart. See ledger ownership.
--ledger-state (env EDGEVPNLEDGERSTATE) is a different thing that gets confused with the above: it points the ledger at a DiskStore (pkg/blockchain/store_disk.go) instead of the default in-memory store. It persists the block chain, not the identity.
Backup, rotation and revocation of a cached key: not addressed anywhere.

Contributions welcome — see contributing.

Run with systemd

Mon, 01 Jan 0001 00:00:00 +0000

This page has not been written. There is no systemd guide on this site beyond the short template-unit section of the install page, which covers only the edgevpn@.service unit that install.sh drops in.

What is missing, and where the source is:

API socket activation. api/api.go (systemdSocketListener) reads LISTEN_PID and LISTEN_FDS and, when LISTEN_PID matches the process and LISTEN_FDS is exactly 1, adopts the already-bound listener on FD 3 instead of binding one itself. The socket’s path, owner, group and mode are then entirely whatever the .socket unit declares — EdgeVPN deliberately does not chmod or unlink it. No example .socket/.service pair is documented anywhere.
APILISTENUNIXMODE. Read by unixSocketMode in api/api.go, it sets the mode only on the path where EdgeVPN creates the socket itself (--api-listen unix:///run/edgevpn.sock). It defaults to 0660 and silently falls back to that default if the value is not valid octal. It has no effect under socket activation.
Hardening a unit. NET_ADMIN, /dev/net/tun, DynamicUser, and which of the environment variables belong in an EnvironmentFile rather than the unit.

Contributions welcome — see contributing.

Tune for low-end devices

Mon, 01 Jan 0001 00:00:00 +0000

This page has not been written. EdgeVPN has a full set of resource-limiting flags and none of them are explained beyond their --help line. There is no guidance on which to reach for on a Raspberry Pi, a router or a container with a tight memory cgroup.

What is missing, and where the source is:

--low-profile (env EDGEVPNLOWPROFILE) is on by default, which is itself undocumented, and its name promises more than it does: in pkg/config/config.go the flag’s only effect is dht.BucketSize(20). A second, unrelated vpn.LowProfile library option in pkg/vpn/config.go swaps in a bounded stream manager (pkg/vpn/vpn.go) and is not wired to the CLI flag. The difference needs writing up.
The ten limit-* flags in cmd/util.go, which configure the libp2p resource manager: --limit-enable (off by default — the others do nothing until it is on), --limit-file, --limit-scope, --limit-config-streams, --limit-config-streams-inbound, --limit-config-streams-outbound, --limit-config-conn, --limit-config-conn-inbound, --limit-config-conn-outbound and --limit-config-fd. Their defaults (200/30/30, 200/30/30, 30) are not documented and their relationship to --limit-scope is not explained.
Connection water marks. --connection-high-water and --connection-low-water (env EDGEVPN_CONNECTION_HIGH_WATER / EDGEVPN_CONNECTION_LOW_WATER) both default to 0, and what 0 means is not stated anywhere. --max-connections (env EDGEVPNMAXCONNS) is a third knob in the same area.
What to turn off. --dht, --mdns, --natservice, --natmap, --autorelay and --relay-service are all on by default and all cost something; --relay-service=false in particular stops the node carrying other peers’ traffic. See relays and hop nodes.

Contributions welcome — see contributing.

How-to guides on EdgeVPN

Run as a VPN

Addressing and DHCP

Static addresses with --address

IPv6

Enable the DNS server

DNS Server

Send and receive files

Sending and receiving files

Sending

Receiving

Tunnel TCP services

Forwarding a local connection

Exposing a service

HTTP egress and the proxy

What this is, and what it is not

Relays and hop nodes

Ledger ownership

Trusted networks

Use EdgeVPN as a library

Run with Docker

Persist node identity and state

Run with systemd

Tune for low-end devices

Static addresses with `--address`