From 0a09a6e6f247729457d15480f8d2b9bb0b89ae5e Mon Sep 17 00:00:00 2001 From: elijah Date: Mon, 29 Aug 2016 22:55:41 -0700 Subject: Updated (very out of date) docs and README.md --- docs/en/tutorials/quick-start.html | 446 +++++++++++++++ docs/en/tutorials/quick-start/index.html | 446 +++++++++++++++ docs/en/tutorials/single-node-email.html | 200 +++++++ docs/en/tutorials/single-node-email/index.html | 200 +++++++ docs/en/tutorials/single-node-vpn.html | 250 +++++++++ docs/en/tutorials/single-node-vpn/index.html | 250 +++++++++ docs/en/tutorials/vagrant.html | 724 +++++++++++++++++++++++++ docs/en/tutorials/vagrant/index.html | 724 +++++++++++++++++++++++++ docs/en/tutorials/vagrant/known-issues.html | 0 docs/en/tutorials/vagrant/quick-start.html | 0 10 files changed, 3240 insertions(+) create mode 100644 docs/en/tutorials/quick-start.html create mode 100644 docs/en/tutorials/quick-start/index.html create mode 100644 docs/en/tutorials/single-node-email.html create mode 100644 docs/en/tutorials/single-node-email/index.html create mode 100644 docs/en/tutorials/single-node-vpn.html create mode 100644 docs/en/tutorials/single-node-vpn/index.html create mode 100644 docs/en/tutorials/vagrant.html create mode 100644 docs/en/tutorials/vagrant/index.html create mode 100644 docs/en/tutorials/vagrant/known-issues.html create mode 100644 docs/en/tutorials/vagrant/quick-start.html (limited to 'docs/en/tutorials') diff --git a/docs/en/tutorials/quick-start.html b/docs/en/tutorials/quick-start.html new file mode 100644 index 00000000..d2670b30 --- /dev/null +++ b/docs/en/tutorials/quick-start.html @@ -0,0 +1,446 @@ + + + + +Quick Start Tutorial - LEAP Platform Documentation + + + + + + + + +
+
+

Quick Start Tutorial

+ +
This tutorial walks you through the initial process of creating and deploying a minimal service provider running the LEAP Platform.
+
+
+ + +

Introduction

+ +

Our goal

+ +

We are going to create a minimal LEAP provider, but one that does not offer any actual services. Check out the other tutorials for adding VPN or email services.

+ +

Our goal is something like this:

+ +
$ leap list
+        NODES   SERVICES          TAGS
+   wildebeest   couchdb, webapp
+
+ +

NOTE: You won’t be able to run that leap list command yet, not until we actually create the node configurations.

+ +

Requirements

+ +
    +
  1. A workstation: This is your local machine that you will run commands on.
    2. +
  2. A server: This is the machine that you will deploy to. The server can be either: + +
      +
    1. A local Vagrant virtual machine: a Vagrant machine can only be useful for testing.
      2. +
    2. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ.
      3. +
    +
    3. +
+ + +

Other things to keep in mind:

+ +
    +
  • The ability to create/modify DNS entries for your domain is preferable, but not needed. If you don’t have access to DNS, you can workaround this by modifying your local resolver, i.e. editing /etc/hosts.
    • +
  • You need to be aware that this process will make changes to your servers, so please be sure that these machines are a basic install with nothing configured or running for other purposes.
    • +
  • Your servers will need to be connected to the internet, and not behind a restrictive firewall.
    • +
+ + +

Prepare your workstation

+ +

In order to be able to manage your servers, you need to install the leap command on your workstation:

+ +

Install pre-requisites

+ +

Install core prerequisites on your workstation.

+ +

Debian & Ubuntu

+ +
workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2
+
+ +

Mac OS

+ +
workstation$ brew install ruby-install
+workstation$ ruby-install ruby
+
+ +

Install the LEAP command-line utility

+ +

Install the leap command system-wide:

+ +
workstation$ sudo gem install leap_cli
+
+ +

Alternately, you can install leap locally without root privileges:

+ +
workstation$ gem install --user-install leap_cli
+workstation$ PATH="$PATH:$(ruby -e 'puts Gem.user_dir')/bin"
+
+ +

If you choose a local install, you probably want to permanently add the –user-install directory to your PATH by adding this to your ~/.profile file (requires logout):

+ +
[ $(which ruby) ] && PATH="$PATH:$(ruby -e 'puts Gem.user_dir')/bin"
+
+ +

To confirm that you installed leap correctly, try running leap --version.

+ +

Create a provider instance

+ +

A provider instance is a directory tree, residing on your workstation, that contains everything you need to manage an infrastructure for a service provider.

+ +

In this case, we create one for example.org and call the instance directory ‘example’.

+ +
workstation$ leap new ~/example
+
+ +

The leap new command will ask you for several required values:

+ +
    +
  • domain: The primary domain name of your service provider. In this tutorial, we will be using “example.org”.
    • +
  • name: The name of your service provider (we use “Example”).
    • +
  • contact emails: A comma separated list of email addresses that should be used for important service provider contacts (for things like postmaster aliases, Tor contact emails, etc).
    • +
  • platform: The directory where you have a copy of the leap_platform git repository checked out. If the platform directory does not yet exist, the leap_platform will be downloaded and placed in that directory.
    • +
+ + +

You could also have passed these configuration options on the command-line, like so:

+ +
workstation$ leap new --contacts your@email.here --domain example.org --name Example --platform=~/leap/leap_platform .
+
+ +

You should now have the following files:

+ +
workstation$ tree example
+example
+├── common.json
+├── Leapfile
+├── nodes/
+├── provider.json
+├── services/
+└── tags/
+
+ +

Now add yourself as a privileged sysadmin who will have access to deploy to servers:

+ +
workstation$ cd example
+workstation$ leap add-user louise --self
+
+ +

Replace “louise” with whatever you want your sysadmin username to be.

+ +

NOTE: Make sure you change directories so that the leap command is run from within the provider instance directory. Most leap commands only work when run from a provider instance.

+ +

Now create the necessary keys and certificates:

+ +
workstation$ leap cert ca
+workstation$ leap cert csr
+
+ +

What do these commands do? The first command will create two Certificate Authorities, one that clients will use to authenticate with the servers and one for backend servers to authenticate with each other. The second command creates a Certificate Signing Request suitable for submission to a commercial CA. It also creates two “dummy” files for you to use temporarily:

+ +
    +
  • files/cert/example.org.crt – This is a “dummy” certificate for your domain that can be used temporarily for testing. Once you get a real certificate from a CA, you should replace this file.
    • +
  • files/cert/commercial_ca.crt – This is “dummy” CA cert the corresponds to the dummy domain certificate. Once you replace the domain certificate, also replace this file with the CA cert from the real Certificate Authority.
    • +
+ + +

If you plan to run a real service provider, see important information on managing keys and certificates.

+ +

Add a node to the provider

+ +

A “node” is a server that is part of your infrastructure. Every node can have one or more services associated with it. We will now add a single node with two services, “webapp” and “couchdb”.

+ +

You have two choices for node type: a real node or a local node.

+ +
    +
  • Real Node: A real node is any physical or paravirtualized server, including KVM, Xen, OpenStack Compute, Amazon EC2, but not VirtualBox or OpenVZ (VirtualBox and OpenVZ use a more limited form of virtualization). The server must be running Debian Jessie.
    • +
  • Local Node: A local node is a virtual machine created by Vagrant, useful for local testing on your workstation.
    • +
+ + +

Getting Vagrant working can be a pain and is covered in other tutorials. If you have a real server available, we suggest you try this tutorial with a real node first.

+ +

Option A: Add a real node

+ +

Note: Installing LEAP Platform on this server will potentially destroy anything you have previously installed on this machine.

+ +

Create a node, with the services “webapp” and “couchdb”:

+ +
workstation$ leap node add wildebeest ip_address:x.x.x.w services:webapp,couchdb
+
+ +

NOTE: replace x.x.x.x with the actual IP address of this server.

+ +

Option B: Add a local node

+ +

Create a node, with the services “webapp” and “couchdb”, and then start the local virtual machine:

+ +
workstation$ leap node add --local wildebeest services:webapp,couchdb
+workstation$ leap local start wildebeest
+
+ +

It will take a while to download the Virtualbox base box and create the virtual machine.

+ +

Deploy your provider

+ +

Initialize the node

+ +

Node initialization only needs to be done once, but there is no harm in doing it multiple times:

+ +
workstation$ leap node init wildebeest
+
+ +

This will initialize the node wildebeest.

+ +

For non-local nodes, when leap node init is run, you will be prompted to verify the fingerprint of the SSH host key and to provide the root password of the server(s). You should only need to do this once.

+ +

Deploy to the node

+ +

The next step is to deploy the LEAP platform to your node. Deployment can take a while to run, especially on the first run, as it needs to update the packages on the new machine.

+ +
workstation$ leap deploy wildebeest
+
+ +

Watch the output for any errors (in red), if everything worked fine, you should now have your first running node. If you do have errors, try doing the deploy again.

+ +

Setup DNS

+ +

The next step is to configure the DNS for your provider. For testing purposes, you can just modify your /etc/hosts file. Please don’t forget about these entries, they will override DNS queries if you setup your DNS later. For a list of what entries to add to /etc/hosts, run this command:

+ +
workstation$ leap compile hosts
+
+ +

Alternately, if you have access to modify the DNS zone entries for your domain:

+ +
workstation$ leap compile zone
+
+ +

NOTE: The resulting zone file is incomplete because it is missing a serial number. Use the output of leap compile zone as a guide, but do not just copy and paste the output. Also, the compile zone output will always exclude mention of local nodes.

+ +

The DNS method will not work for local nodes created with Vagrant.

+ +

Test that things worked correctly

+ +

To run troubleshooting tests:

+ +
workstation$ leap test
+
+ +

Alternately, you can run these same tests from the server itself:

+ +
workstation$ leap ssh wildebeest
+wildebeest# run_tests
+
+ +

Create an administrator

+ +

Assuming that you set up your DNS or /etc/hosts file, you should be able to load https://example.org in your web browser (where example.org is whatever domain name you actually used).

+ +

Your browser will complain about an untrusted cert, but for now just bypass this. From there, you should be able to register a new user and login.

+ +

Once you have created a user, you can now make this user an administrator. For example, if you created a user kangaroo, you would create the file services/webapp.json with the following content:

+ +
{
+    "webapp": {
+        "admins": ["kangaroo"]
+    }
+}
+
+ +

Save that file and run leap deploy again. When you next log on to the web application, the user kangaroo will now be an admin.

+ +

If you want to restrict who can register a new user, see webapp for configuration options.

+ +

What is next?

+ +

Add an end-user service

+ +

You should now have a minimal service provider with a single node. This service provider is pointless at the moment, because it does not include any end-user services like VPN or email. To add one of these services, continue with one of the following tutorials:

+ + + + +

Learn more

+ +

We have only just scratched the surface of the possible ways to configure and deploy your service provider. Your next step should be:

+ + + + +
+
+ + diff --git a/docs/en/tutorials/quick-start/index.html b/docs/en/tutorials/quick-start/index.html new file mode 100644 index 00000000..27b21238 --- /dev/null +++ b/docs/en/tutorials/quick-start/index.html @@ -0,0 +1,446 @@ + + + + +Quick Start Tutorial - LEAP Platform Documentation + + + + + + + + +
+
+

Quick Start Tutorial

+ +
This tutorial walks you through the initial process of creating and deploying a minimal service provider running the LEAP Platform.
+
+
+ + +

Introduction

+ +

Our goal

+ +

We are going to create a minimal LEAP provider, but one that does not offer any actual services. Check out the other tutorials for adding VPN or email services.

+ +

Our goal is something like this:

+ +
$ leap list
+        NODES   SERVICES          TAGS
+   wildebeest   couchdb, webapp
+
+ +

NOTE: You won’t be able to run that leap list command yet, not until we actually create the node configurations.

+ +

Requirements

+ +
    +
  1. A workstation: This is your local machine that you will run commands on.
    2. +
  2. A server: This is the machine that you will deploy to. The server can be either: + +
      +
    1. A local Vagrant virtual machine: a Vagrant machine can only be useful for testing.
      2. +
    2. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ.
      3. +
    +
    3. +
+ + +

Other things to keep in mind:

+ +
    +
  • The ability to create/modify DNS entries for your domain is preferable, but not needed. If you don’t have access to DNS, you can workaround this by modifying your local resolver, i.e. editing /etc/hosts.
    • +
  • You need to be aware that this process will make changes to your servers, so please be sure that these machines are a basic install with nothing configured or running for other purposes.
    • +
  • Your servers will need to be connected to the internet, and not behind a restrictive firewall.
    • +
+ + +

Prepare your workstation

+ +

In order to be able to manage your servers, you need to install the leap command on your workstation:

+ +

Install pre-requisites

+ +

Install core prerequisites on your workstation.

+ +

Debian & Ubuntu

+ +
workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2
+
+ +

Mac OS

+ +
workstation$ brew install ruby-install
+workstation$ ruby-install ruby
+
+ +

Install the LEAP command-line utility

+ +

Install the leap command system-wide:

+ +
workstation$ sudo gem install leap_cli
+
+ +

Alternately, you can install leap locally without root privileges:

+ +
workstation$ gem install --user-install leap_cli
+workstation$ PATH="$PATH:$(ruby -e 'puts Gem.user_dir')/bin"
+
+ +

If you choose a local install, you probably want to permanently add the –user-install directory to your PATH by adding this to your ~/.profile file (requires logout):

+ +
[ $(which ruby) ] && PATH="$PATH:$(ruby -e 'puts Gem.user_dir')/bin"
+
+ +

To confirm that you installed leap correctly, try running leap --version.

+ +

Create a provider instance

+ +

A provider instance is a directory tree, residing on your workstation, that contains everything you need to manage an infrastructure for a service provider.

+ +

In this case, we create one for example.org and call the instance directory ‘example’.

+ +
workstation$ leap new ~/example
+
+ +

The leap new command will ask you for several required values:

+ +
    +
  • domain: The primary domain name of your service provider. In this tutorial, we will be using “example.org”.
    • +
  • name: The name of your service provider (we use “Example”).
    • +
  • contact emails: A comma separated list of email addresses that should be used for important service provider contacts (for things like postmaster aliases, Tor contact emails, etc).
    • +
  • platform: The directory where you have a copy of the leap_platform git repository checked out. If the platform directory does not yet exist, the leap_platform will be downloaded and placed in that directory.
    • +
+ + +

You could also have passed these configuration options on the command-line, like so:

+ +
workstation$ leap new --contacts your@email.here --domain example.org --name Example --platform=~/leap/leap_platform .
+
+ +

You should now have the following files:

+ +
workstation$ tree example
+example
+├── common.json
+├── Leapfile
+├── nodes/
+├── provider.json
+├── services/
+└── tags/
+
+ +

Now add yourself as a privileged sysadmin who will have access to deploy to servers:

+ +
workstation$ cd example
+workstation$ leap add-user louise --self
+
+ +

Replace “louise” with whatever you want your sysadmin username to be.

+ +

NOTE: Make sure you change directories so that the leap command is run from within the provider instance directory. Most leap commands only work when run from a provider instance.

+ +

Now create the necessary keys and certificates:

+ +
workstation$ leap cert ca
+workstation$ leap cert csr
+
+ +

What do these commands do? The first command will create two Certificate Authorities, one that clients will use to authenticate with the servers and one for backend servers to authenticate with each other. The second command creates a Certificate Signing Request suitable for submission to a commercial CA. It also creates two “dummy” files for you to use temporarily:

+ +
    +
  • files/cert/example.org.crt – This is a “dummy” certificate for your domain that can be used temporarily for testing. Once you get a real certificate from a CA, you should replace this file.
    • +
  • files/cert/commercial_ca.crt – This is “dummy” CA cert the corresponds to the dummy domain certificate. Once you replace the domain certificate, also replace this file with the CA cert from the real Certificate Authority.
    • +
+ + +

If you plan to run a real service provider, see important information on managing keys and certificates.

+ +

Add a node to the provider

+ +

A “node” is a server that is part of your infrastructure. Every node can have one or more services associated with it. We will now add a single node with two services, “webapp” and “couchdb”.

+ +

You have two choices for node type: a real node or a local node.

+ +
    +
  • Real Node: A real node is any physical or paravirtualized server, including KVM, Xen, OpenStack Compute, Amazon EC2, but not VirtualBox or OpenVZ (VirtualBox and OpenVZ use a more limited form of virtualization). The server must be running Debian Jessie.
    • +
  • Local Node: A local node is a virtual machine created by Vagrant, useful for local testing on your workstation.
    • +
+ + +

Getting Vagrant working can be a pain and is covered in other tutorials. If you have a real server available, we suggest you try this tutorial with a real node first.

+ +

Option A: Add a real node

+ +

Note: Installing LEAP Platform on this server will potentially destroy anything you have previously installed on this machine.

+ +

Create a node, with the services “webapp” and “couchdb”:

+ +
workstation$ leap node add wildebeest ip_address:x.x.x.w services:webapp,couchdb
+
+ +

NOTE: replace x.x.x.x with the actual IP address of this server.

+ +

Option B: Add a local node

+ +

Create a node, with the services “webapp” and “couchdb”, and then start the local virtual machine:

+ +
workstation$ leap node add --local wildebeest services:webapp,couchdb
+workstation$ leap local start wildebeest
+
+ +

It will take a while to download the Virtualbox base box and create the virtual machine.

+ +

Deploy your provider

+ +

Initialize the node

+ +

Node initialization only needs to be done once, but there is no harm in doing it multiple times:

+ +
workstation$ leap node init wildebeest
+
+ +

This will initialize the node wildebeest.

+ +

For non-local nodes, when leap node init is run, you will be prompted to verify the fingerprint of the SSH host key and to provide the root password of the server(s). You should only need to do this once.

+ +

Deploy to the node

+ +

The next step is to deploy the LEAP platform to your node. Deployment can take a while to run, especially on the first run, as it needs to update the packages on the new machine.

+ +
workstation$ leap deploy wildebeest
+
+ +

Watch the output for any errors (in red), if everything worked fine, you should now have your first running node. If you do have errors, try doing the deploy again.

+ +

Setup DNS

+ +

The next step is to configure the DNS for your provider. For testing purposes, you can just modify your /etc/hosts file. Please don’t forget about these entries, they will override DNS queries if you setup your DNS later. For a list of what entries to add to /etc/hosts, run this command:

+ +
workstation$ leap compile hosts
+
+ +

Alternately, if you have access to modify the DNS zone entries for your domain:

+ +
workstation$ leap compile zone
+
+ +

NOTE: The resulting zone file is incomplete because it is missing a serial number. Use the output of leap compile zone as a guide, but do not just copy and paste the output. Also, the compile zone output will always exclude mention of local nodes.

+ +

The DNS method will not work for local nodes created with Vagrant.

+ +

Test that things worked correctly

+ +

To run troubleshooting tests:

+ +
workstation$ leap test
+
+ +

Alternately, you can run these same tests from the server itself:

+ +
workstation$ leap ssh wildebeest
+wildebeest# run_tests
+
+ +

Create an administrator

+ +

Assuming that you set up your DNS or /etc/hosts file, you should be able to load https://example.org in your web browser (where example.org is whatever domain name you actually used).

+ +

Your browser will complain about an untrusted cert, but for now just bypass this. From there, you should be able to register a new user and login.

+ +

Once you have created a user, you can now make this user an administrator. For example, if you created a user kangaroo, you would create the file services/webapp.json with the following content:

+ +
{
+    "webapp": {
+        "admins": ["kangaroo"]
+    }
+}
+
+ +

Save that file and run leap deploy again. When you next log on to the web application, the user kangaroo will now be an admin.

+ +

If you want to restrict who can register a new user, see webapp for configuration options.

+ +

What is next?

+ +

Add an end-user service

+ +

You should now have a minimal service provider with a single node. This service provider is pointless at the moment, because it does not include any end-user services like VPN or email. To add one of these services, continue with one of the following tutorials:

+ + + + +

Learn more

+ +

We have only just scratched the surface of the possible ways to configure and deploy your service provider. Your next step should be:

+ + + + +
+
+ + diff --git a/docs/en/tutorials/single-node-email.html b/docs/en/tutorials/single-node-email.html new file mode 100644 index 00000000..6678fec3 --- /dev/null +++ b/docs/en/tutorials/single-node-email.html @@ -0,0 +1,200 @@ + + + + +Quick email - LEAP Platform Documentation + + + + + + + + +
+
+

Single node email tutorial

+ +
Tutorial for setting up a simple email provider.
+
+
+ + +

This tutorial walks you through the initial process of creating and deploying a minimal email service provider. Please first complete the Quick Start Tutorial. This tutorial will pick up where that one left off.

+ +

Our goal

+ +

We are going to create a minimal LEAP provider offering email service.

+ +

Our goal is something like this:

+ +
$ leap list
+    NODES       SERVICES                       TAGS
+    wildebeest  couchdb, mx, soledad, webapp
+
+ +

Where ‘wildebeest’ is whatever name you chose for your node in the Quick Start Tutorial.

+ +

Add email services to the node

+ +

In order to add services to a node, edit the node’s JSON configuration file.

+ +

In our example, we would edit nodes/wildebeest.json:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "mx", "soledad"]
+}
+
+ +

Here, we added mx and soledad to the node’s services list. Briefly:

+ +
    +
  • mx: nodes with the mx service will run postfix mail transfer agent, and are able to receive and relay email on behalf of your domain. You can have as many as you want, spread out over as many nodes as you want.
    • +
  • soledad: nodes with soledad service run the server-side daemon that allows the client to synchronize a user’s personal data store among their devices. Currently, soledad only runs on nodes that are also couchdb nodes.
    • +
+ + +

For more details, see the Services overview, or the individual pages for the mx and soledad services.

+ +

Deploy to the node

+ +

Now you should deploy to your node.

+ +
workstation$ leap deploy
+
+ +

Setup DNS

+ +

There are several important DNS entries that all email providers should have:

+ +
    +
  • SPF (Sender Policy Framework): With SPF, an email provider advertises in their DNS which servers should be allowed to relay email on behalf of your domain.
    • +
  • DKIM (DomainKey Identified Mail): With DKIM, an email provider is able to vouch for the validity of certain headers in outgoing mail, allowing the receiving provider to have more confidence in these values when processing the message for spam or abuse.
    • +
+ + +

In order to take advantage of SPF and DKIM, run this command:

+ +
workstation$ leap compile zone
+
+ +

Then take the output of that command and merge it with the DNS zone file for your domain.

+ +

CAUTION: the output of leap compile zone is not a complete zone file since it is missing a serial number. You will need to manually merge it with your existing zone file.

+ +

Test it out

+ +

First, run:

+ +
workstation# leap test
+
+ +

Then fire up the bitmask client, register a new user with your provider, and try sending and receiving email.

+ +
+
+ + diff --git a/docs/en/tutorials/single-node-email/index.html b/docs/en/tutorials/single-node-email/index.html new file mode 100644 index 00000000..45a1264f --- /dev/null +++ b/docs/en/tutorials/single-node-email/index.html @@ -0,0 +1,200 @@ + + + + +Quick email - LEAP Platform Documentation + + + + + + + + +
+
+

Single node email tutorial

+ +
Tutorial for setting up a simple email provider.
+
+
+ + +

This tutorial walks you through the initial process of creating and deploying a minimal email service provider. Please first complete the Quick Start Tutorial. This tutorial will pick up where that one left off.

+ +

Our goal

+ +

We are going to create a minimal LEAP provider offering email service.

+ +

Our goal is something like this:

+ +
$ leap list
+    NODES       SERVICES                       TAGS
+    wildebeest  couchdb, mx, soledad, webapp
+
+ +

Where ‘wildebeest’ is whatever name you chose for your node in the Quick Start Tutorial.

+ +

Add email services to the node

+ +

In order to add services to a node, edit the node’s JSON configuration file.

+ +

In our example, we would edit nodes/wildebeest.json:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "mx", "soledad"]
+}
+
+ +

Here, we added mx and soledad to the node’s services list. Briefly:

+ +
    +
  • mx: nodes with the mx service will run postfix mail transfer agent, and are able to receive and relay email on behalf of your domain. You can have as many as you want, spread out over as many nodes as you want.
    • +
  • soledad: nodes with soledad service run the server-side daemon that allows the client to synchronize a user’s personal data store among their devices. Currently, soledad only runs on nodes that are also couchdb nodes.
    • +
+ + +

For more details, see the Services overview, or the individual pages for the mx and soledad services.

+ +

Deploy to the node

+ +

Now you should deploy to your node.

+ +
workstation$ leap deploy
+
+ +

Setup DNS

+ +

There are several important DNS entries that all email providers should have:

+ +
    +
  • SPF (Sender Policy Framework): With SPF, an email provider advertises in their DNS which servers should be allowed to relay email on behalf of your domain.
    • +
  • DKIM (DomainKey Identified Mail): With DKIM, an email provider is able to vouch for the validity of certain headers in outgoing mail, allowing the receiving provider to have more confidence in these values when processing the message for spam or abuse.
    • +
+ + +

In order to take advantage of SPF and DKIM, run this command:

+ +
workstation$ leap compile zone
+
+ +

Then take the output of that command and merge it with the DNS zone file for your domain.

+ +

CAUTION: the output of leap compile zone is not a complete zone file since it is missing a serial number. You will need to manually merge it with your existing zone file.

+ +

Test it out

+ +

First, run:

+ +
workstation# leap test
+
+ +

Then fire up the bitmask client, register a new user with your provider, and try sending and receiving email.

+ +
+
+ + diff --git a/docs/en/tutorials/single-node-vpn.html b/docs/en/tutorials/single-node-vpn.html new file mode 100644 index 00000000..1bfeb937 --- /dev/null +++ b/docs/en/tutorials/single-node-vpn.html @@ -0,0 +1,250 @@ + + + + +Quick VPN - LEAP Platform Documentation + + + + + + + + +
+
+

Single node VPN tutorial

+ +
Tutorial for setting up a simple VPN provider.
+
+
+ + +

This tutorial walks you through the initial process of creating and deploying a minimal VPN service provider. Please first complete the Quick Start Tutorial. This tutorial will pick up where that one left off.

+ +

NOTE: For the VPN to work, you must use a real or paravirtualized node, not a local Vagrant node.

+ +

Our goal

+ +

We are going to create a minimal LEAP provider offering VPN service.

+ +

Our goal is something like this:

+ +
$ leap list
+    NODES       SERVICES                       TAGS
+    wildebeest  couchdb, webapp, openvpn, tor
+
+ +

Where ‘wildebeest’ is whatever name you chose for your node in the Quick Start Tutorial.

+ +

Add VPN service to the node

+ +

In order to add services to a node, edit the node’s JSON configuration file.

+ +

In our example, we would edit nodes/wildebeest.json:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "openvpn", "tor"]
+}
+
+ +

Here, we added openvpn and tor to the node’s services list. Briefly:

+ +
    +
  • openvpn: nodes with the openvpn service will become OpenVPN gateways that clients connect to in order to proxy their internet connection. You can have as many as you want, spread out over as many nodes as you want.
    • +
  • tor: nodes with tor service become Tor exit nodes. This is entirely optional, and will add additional bandwidth to your node. If you don’t have many VPN users, the added traffic will help create cover traffic for your users. On the down side, this VPN gateway will get flagged as an anonymous proxy and some sites may block traffic from it.
    • +
+ + +

For more details, see the Services overview, or the individual pages for the openvpn and tor services.

+ +

Add gateway_address to the node

+ +

VPN gateways require two different IP addresses:

+ +
    +
  • ip_address: This property is used for VPN traffic egress. In other words, all VPN traffic appears to come from this IP address. This is also the main IP of the server.
    • +
  • openvpn.gateway_address: This property is used for VPN traffic ingress. In other words, clients will connect to this IP address.
    • +
+ + +

The node configuration file should now look like this:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "openvpn", "tor"],
+  "openvpn": {
+    "gateway_address": "2.2.2.2"
+  }
+}
+
+ +

Why two different addresses? Without this, the traffic from one VPN user to another would not be encrypted. This is because the routing table of VPN clients must ensure that packets with a destination of the VPN gateway are sent unmodified and don’t get passed through the VPN’s encryption.

+ +

Generate a Diffie-Hellman file

+ +

Next we need to create a Diffie-Hellman parameter file, used for forward secret OpenVPN ciphers. You only need to do this once.

+ +
workstation$ leap cert dh
+
+ +

Feel free to erase the resulting DH file and regenerate it as you please.

+ +

Deploy to the node

+ +

Now you should deploy to your node. This may take a while.

+ +
workstation$ leap deploy
+
+ +

If the deploy was not successful, try to run it again.

+ +

Test it out

+ +

First, run:

+ +
workstation$ leap test
+
+ +

Then fire up the Bitmask client, register a new user with your provider, and turn on the VPN connection.

+ +

Alternately, you can also manually connect to your VPN gateway using OpenVPN on the command line:

+ +
workstation$ sudo apt install openvpn
+workstation$ leap test init
+workstation$ sudo openvpn --config test/openvpn/default_unlimited.ovpn
+
+ +

Make sure that Bitmask is not connected to the VPN when you run that command.

+ +

The name of the test configuration might differ depending on your setup. The test configuration created by leap test init includes a client certificate that will expire, so you may need to re-run leap test init if it has been a while since you last generated the test configuration.

+ +

What do do next

+ +

A VPN provider with a single gateway is kind of limited. You can add as many nodes with service openvpn as you like. There is no communication among the VPN gateways or with the webapp or couchdb nodes, so there is no issue with scaling out the number of gateways.

+ +

For example, add some more nodes:

+ +
workstation$ leap node add giraffe ip_address:1.1.1.2 services:openvpn openvpn.gateway_address:2.2.2.3
+workstation$ leap node add rhino ip_address:1.1.1.3 services:openvpn openvpn.gateway_address:2.2.2.4
+workstation$ leap node init giraffe rhino
+workstation$ leap deploy
+
+ +

Now you have three VPN gateways.

+ +

One consideration is that you should tag each VPN gateway with a location. This helps the client determine which VPN gateway it should connect to by default and will allow the user to choose among gateways based on location.

+ +
+
+ + diff --git a/docs/en/tutorials/single-node-vpn/index.html b/docs/en/tutorials/single-node-vpn/index.html new file mode 100644 index 00000000..adceb66f --- /dev/null +++ b/docs/en/tutorials/single-node-vpn/index.html @@ -0,0 +1,250 @@ + + + + +Quick VPN - LEAP Platform Documentation + + + + + + + + +
+
+

Single node VPN tutorial

+ +
Tutorial for setting up a simple VPN provider.
+
+
+ + +

This tutorial walks you through the initial process of creating and deploying a minimal VPN service provider. Please first complete the Quick Start Tutorial. This tutorial will pick up where that one left off.

+ +

NOTE: For the VPN to work, you must use a real or paravirtualized node, not a local Vagrant node.

+ +

Our goal

+ +

We are going to create a minimal LEAP provider offering VPN service.

+ +

Our goal is something like this:

+ +
$ leap list
+    NODES       SERVICES                       TAGS
+    wildebeest  couchdb, webapp, openvpn, tor
+
+ +

Where ‘wildebeest’ is whatever name you chose for your node in the Quick Start Tutorial.

+ +

Add VPN service to the node

+ +

In order to add services to a node, edit the node’s JSON configuration file.

+ +

In our example, we would edit nodes/wildebeest.json:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "openvpn", "tor"]
+}
+
+ +

Here, we added openvpn and tor to the node’s services list. Briefly:

+ +
    +
  • openvpn: nodes with the openvpn service will become OpenVPN gateways that clients connect to in order to proxy their internet connection. You can have as many as you want, spread out over as many nodes as you want.
    • +
  • tor: nodes with tor service become Tor exit nodes. This is entirely optional, and will add additional bandwidth to your node. If you don’t have many VPN users, the added traffic will help create cover traffic for your users. On the down side, this VPN gateway will get flagged as an anonymous proxy and some sites may block traffic from it.
    • +
+ + +

For more details, see the Services overview, or the individual pages for the openvpn and tor services.

+ +

Add gateway_address to the node

+ +

VPN gateways require two different IP addresses:

+ +
    +
  • ip_address: This property is used for VPN traffic egress. In other words, all VPN traffic appears to come from this IP address. This is also the main IP of the server.
    • +
  • openvpn.gateway_address: This property is used for VPN traffic ingress. In other words, clients will connect to this IP address.
    • +
+ + +

The node configuration file should now look like this:

+ +
{
+  "ip_address": "1.1.1.1",
+  "services": ["couchdb", "webapp", "openvpn", "tor"],
+  "openvpn": {
+    "gateway_address": "2.2.2.2"
+  }
+}
+
+ +

Why two different addresses? Without this, the traffic from one VPN user to another would not be encrypted. This is because the routing table of VPN clients must ensure that packets with a destination of the VPN gateway are sent unmodified and don’t get passed through the VPN’s encryption.

+ +

Generate a Diffie-Hellman file

+ +

Next we need to create a Diffie-Hellman parameter file, used for forward secret OpenVPN ciphers. You only need to do this once.

+ +
workstation$ leap cert dh
+
+ +

Feel free to erase the resulting DH file and regenerate it as you please.

+ +

Deploy to the node

+ +

Now you should deploy to your node. This may take a while.

+ +
workstation$ leap deploy
+
+ +

If the deploy was not successful, try to run it again.

+ +

Test it out

+ +

First, run:

+ +
workstation$ leap test
+
+ +

Then fire up the Bitmask client, register a new user with your provider, and turn on the VPN connection.

+ +

Alternately, you can also manually connect to your VPN gateway using OpenVPN on the command line:

+ +
workstation$ sudo apt install openvpn
+workstation$ leap test init
+workstation$ sudo openvpn --config test/openvpn/default_unlimited.ovpn
+
+ +

Make sure that Bitmask is not connected to the VPN when you run that command.

+ +

The name of the test configuration might differ depending on your setup. The test configuration created by leap test init includes a client certificate that will expire, so you may need to re-run leap test init if it has been a while since you last generated the test configuration.

+ +

What do do next

+ +

A VPN provider with a single gateway is kind of limited. You can add as many nodes with service openvpn as you like. There is no communication among the VPN gateways or with the webapp or couchdb nodes, so there is no issue with scaling out the number of gateways.

+ +

For example, add some more nodes:

+ +
workstation$ leap node add giraffe ip_address:1.1.1.2 services:openvpn openvpn.gateway_address:2.2.2.3
+workstation$ leap node add rhino ip_address:1.1.1.3 services:openvpn openvpn.gateway_address:2.2.2.4
+workstation$ leap node init giraffe rhino
+workstation$ leap deploy
+
+ +

Now you have three VPN gateways.

+ +

One consideration is that you should tag each VPN gateway with a location. This helps the client determine which VPN gateway it should connect to by default and will allow the user to choose among gateways based on location.

+ +
+
+ + diff --git a/docs/en/tutorials/vagrant.html b/docs/en/tutorials/vagrant.html new file mode 100644 index 00000000..3d4f0520 --- /dev/null +++ b/docs/en/tutorials/vagrant.html @@ -0,0 +1,724 @@ + + + + +Vagrant - LEAP Platform Documentation + + + + + + + + +
+
+

Vagrant and the LEAP Platform

+ +
Running a local provider with Vagrant
+
+
+ + +

What is Vagrant?

+ +

Vagrant is a tool to make it easier to manage virtual machines running on your desktop computer (typically for testing or development purposes). You can use Vagrant to create virtual machines and deploy the LEAP platform locally.

+ +

Vagrant can be a pain to get working initially, but this page should help you get through the process. Please make sure you have at least Vagrant v1.5 installed.

+ +

There are two ways you can setup LEAP platform using Vagrant.

+ +
    +
  1. use the leap command: this will allow you to create multiple virtual machines.
    2. +
  2. use static Vagrantfile: there is a static Vagrantfile that is distributed with the leap_platform.git. This only supports a single, pre-configured virtual machine, but can get you started more quickly.
    3. +
+ + +

Install Vagrant

+ +

Requirements:

+ +
    +
  • A real machine with virtualization support in the CPU (VT-x or AMD-V). In other words, not a virtual machine.
    • +
  • Have at least 4gb of RAM.
    • +
  • Have a fast internet connection (because you will be downloading a lot of big files, like virtual machine images).
    • +
  • You should do everything described below as an unprivileged user, and only run those commands as root that are noted with sudo in front of them. Other than those commands, there is no need for privileged access to your machine, and in fact things may not work correctly.
    • +
+ + +

Debian & Ubuntu

+ +

Install core prerequisites:

+ +
sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make
+
+ +

Install Vagrant:

+ +
sudo apt-get install vagrant virtualbox
+
+ +

If you want to use libvirt instead of virtualbox, you don’t need to install virtualbox. See support for libvirt.

+ +

Mac OS X 10.9 (Mavericks)

+ +

Install Homebrew package manager from http://brew.sh/ and enable the System Duplicates Repository (needed to update old software versions delivered by Apple) with

+ +
brew tap homebrew/dupes
+
+ +

Update OpenSSH to support ECDSA keys. Follow this guide to let your system use the Homebrew binary.

+ +
brew install openssh --with-brewed-openssl --with-keychain-support
+
+ +

The certtool provided by Apple it’s really old, install the one provided by GnuTLS and shadow the system’s default.

+ +
sudo brew install gnutls
+ln -sf /usr/local/bin/gnutls-certtool /usr/local/bin/certool
+
+ +

Install the Vagrant and VirtualBox packages for OS X from their respective Download pages.

+ + + + +

Vagrant with leap command

+ +

If you have not done so, install leap command line tool:

+ +
gem install leap_cli
+
+ +

Creating local nodes

+ +

When you create a service provider, your servers are called “nodes”. When a node is virtual and exists only locally using vagrant, this type of node is called a “local node”.

+ +

If you do not have a provider already, you will need to create one and configure it before continuing (see the Quick Start guide).

+ +

These commands, for example, will create an initial provider directory “myprovider”:

+ +
$ leap new --domain example.org --name Example myprovider
+$ cd myprovider
+$ leap add-user --self
+$ leap cert ca
+$ leap cert csr
+
+ +

To create local nodes, add the flag --local to the leap node add command. For example:

+ +
$ leap node add --local web1 services:webapp
+ = created nodes/web1.json
+ = created files/nodes/web1/
+ = created files/nodes/web1/web1.key
+ = created files/nodes/web1/web1.crt
+
+ +

This command creates a node configuration file in nodes/web1.json with the webapp service.

+ +

Starting local nodes

+ +

In order to test the node “web1” we need to start it. Starting a node for the first time will spin up a virtual machine. The first time you do this will take some time because it will need to download a VM image (about 700mb). After you’ve downloaded the base image, you will not need to download it again, and instead you will re-use the downloaded image (until you need to update the image).

+ +

NOTE: Many people have difficulties getting Vagrant working. If the following commands do not work, please see the troubleshooting section below.

+ +
$ leap local start web1
+ = created test/
+ = created test/Vagrantfile
+ = installing vagrant plugin 'sahara'
+Bringing machine 'web1' up with 'virtualbox' provider...
+[web1] Box 'leap-jessie' was not found. Fetching box from specified URL for
+the provider 'virtualbox'. Note that if the URL does not have
+a box for this provider, you should interrupt Vagrant now and add
+the box yourself. Otherwise Vagrant will attempt to download the
+full box prior to discovering this error.
+Downloading or copying the box...
+Progress: 3% (Rate: 560k/s, Estimated time remaining: 0:13:36)
+...
+Bringing machine 'web1' up with 'virtualbox' provider...
+[web1] Importing base box 'leap-jessie'...
+0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
+
+ +

Now the virtual machine ‘web1’ is running. You can add another local node using the same process. For example, the webapp node needs a databasse to run, so let’s add a “couchdb” node:

+ +
$ leap node add --local db1 services:couchdb
+$ leap local start
+ = updated test/Vagrantfile
+Bringing machine 'db1' up with 'virtualbox' provider...
+[db1] Importing base box 'leap-jessie'...
+[db1] Matching MAC address for NAT networking...
+[db1] Setting the name of the VM...
+[db1] Clearing any previously set forwarded ports...
+[db1] Fixed port collision for 22 => 2222. Now on port 2202.
+[db1] Creating shared folders metadata...
+[db1] Clearing any previously set network interfaces...
+[db1] Preparing network interfaces based on configuration...
+[db1] Forwarding ports...
+[db1] -- 22 => 2202 (adapter 1)
+[db1] Running any VM customizations...
+[db1] Booting VM...
+[db1] Waiting for VM to boot. This can take a few minutes.
+[db1] VM booted and ready for use!
+[db1] Configuring and enabling network interfaces...
+[db1] Mounting shared folders...
+[db1] -- /vagrant
+
+ +

You now can follow the normal LEAP process and initialize it and then deploy your recipes to it:

+ +
$ leap node init web1
+$ leap deploy web1
+$ leap node init db1
+$ leap deploy db1
+
+ +

Useful local commands

+ +

There are many useful things you can do with a virtualized development environment.

+ +

Listing what machines are running

+ +

Now you have the two virtual machines “web1” and “db1” running, you can see the running machines as follows:

+ +
$ leap local status
+Current machine states:
+
+db1                      running (virtualbox)
+web1                     running (virtualbox)
+
+This environment represents multiple VMs. The VMs are all listed
+above with their current state. For more information about a specific
+VM, run `vagrant status NAME`.
+
+ +

Stopping machines

+ +

It is not recommended that you leave your virtual machines running when you are not using them. They consume memory and other resources! To stop your machines, simply do the following:

+ +
$ leap local stop web1 db1
+
+ +

Connecting to machines

+ +

You can connect to your local nodes just like you do with normal LEAP nodes, by running ‘leap ssh node’.

+ +

However, if you cannot connect to your local node, because the networking is not setup properly, or you have deployed a firewall that locks you out, you may need to access the graphical console.

+ +

In order to do that, you will need to configure Vagrant to launch a graphical console and then you can login as root there to diagnose the networking problem. To do this, add the following to your $HOME/.leaprc:

+ +
@custom_vagrant_vm_line = 'config.vm.provider "virtualbox" do |v|
+  v.gui = true
+end'
+
+ +

and then start, or restart, your local Vagrant node. You should get a VirtualBox graphical interface presented to you showing you the bootup and eventually the login.

+ +

Snapshotting machines

+ +

A very useful feature of local Vagrant development nodes is the ability to snapshot the current state and then revert to that when you need.

+ +

For example, perhaps the base image is a little bit out of date and you want to get the packages updated to the latest before continuing. You can do that simply by starting the node, connecting to it and updating the packages and then snapshotting the node:

+ +
$ leap local start web1
+$ leap ssh web1
+web1# apt-get -u dist-upgrade
+web1# exit
+$ leap local save web1
+
+ +

Now you can deploy to web1 and if you decide you want to revert to the state before deployment, you simply have to reset the node to your previous save:

+ +
$ leap local reset web1
+
+ +

More information

+ +

See leap help local for a complete list of local-only commands and how they can be used.

+ +

2. Vagrant with static Vagrantfile

+ +

You can use the static Vagrantfile if you want to get up a running with a pre-canned test provider.

+ +

It will install a single node mail server in the default configuration with one single command.

+ +

Clone the platform with

+ +
git clone --recursive -b develop https://github.com/leapcode/leap_platform.git
+
+ +

Start the vagrant box with

+ +
cd leap_platform
+vagrant up
+
+ +

Follow the instructions how to configure your /etc/hosts +in order to use the provider!

+ +

You can login via ssh with the systemuser vagrant and the same password.

+ +
vagrant ssh
+
+ +

On the host, run the tests to check if everything is working as expected:

+ +
cd /home/vagrant/leap/configuration/
+leap test
+
+ +

Use the bitmask client to do an initial soledad sync

+ +

Copy the self-signed CA certificate from the host. +The easiest way is to use the vagrant-scp plugin:

+ +
vagrant scp :/home/vagrant/leap/configuration/files/ca/ca.crt /tmp/example.org.ca.crt
+
+vagrant@node1:~/leap/configuration$ cat files/ca/ca.crt
+
+ +

and write it into a file, needed by the bitmask client:

+ +
bitmask --ca-cert-file /tmp/example.org.ca.crt
+
+ +

On the first run, bitmask is creating a gpg keypair. This is +needed for delivering and encrypting incoming mails.

+ +

Testing email

+ +
sudo apt install swaks
+swaks -f test22@leap.se -t test22@example.org -s example.org
+
+ +

check the logs:

+ +
sudo less /var/log/mail.log
+sudo less /var/log/leap/mx.log
+
+ +

if an error occurs, see if the mail is still laying in the mailspool dir:

+ +
sudo ls /var/mail/leap-mx/Maildir/new
+
+ +

Re-run bitmask client to sync your mail

+ +
bitmask --ca-cert-file /tmp/example.org.ca.crt
+
+ +

Now, connect your favorite mail client to the imap and smtp proxy +started by the bitmask client:

+ +
https://bitmask.net/en/help/email
+
+ +

Happy testing !

+ +

Using the Webapp

+ +

There are 2 users preconfigured:

+ +

. testuser with pw hallo123 +. testadmin with pw hallo123

+ +

login as testadmin to access the webapp with admin priviledges.

+ +

Support for libvirt

+ +

Install libvirt plugin

+ +

By default, Vagrant will use VirtualBox to create the virtual machines, but this is how you can use libvirt. Using libvirt is more efficient, but VirtualBox is more stable and easier to set up.

+ +

For debian/ubuntu:

+ +
sudo apt-get install libvirt-bin libvirt-dev
+
+# to build the vagrant-libvirt plugin you need the following packages:
+sudo apt-get install ruby-dev libxslt-dev libxml2-dev libvirt-dev
+
+# install the required plugins
+vagrant plugin install vagrant-libvirt fog fog-libvirt sahara
+
+ +

Log out and then log back in.

+ +

Note: if running ubuntu 15.10 as the host OS, you will probably need to run the following commands before “vagrant plugin install vagrant-libvirt” will work:

+ +
ln -sf /usr/lib/liblzma.so.5 /opt/vagrant/embedded/lib
+ln -sf /usr/lib/liblzma.so.5.0.0 /opt/vagrant/embedded/lib
+
+ +

Create libvirt pool

+ +

Next, you must create the libvirt image pool. The “default” pool uses /var/lib/libvirt/images, but Vagrant will not download base boxes there. Instead, create a libvirt pool called “vagrant”, like so:

+ +
virsh pool-define-as vagrant dir - - - - /home/$USER/.vagrant.d/boxes
+virsh pool-start vagrant
+virsh pool-autostart vagrant
+
+ +

If you want to use a name different than “vagrant” for the pool, you can change the name in Leapfile by setting the @vagrant_libvirt_pool variable:

+ +
@vagrant_libvirt_pool = "vagrant"
+
+ +

Force use of libvirt

+ +

Finally, you need to tell Vagrant to use libvirt instead of VirtualBox. If using vagrant with leap_cli, modify your Leapfile or .leaprc file and add this line:

+ +
@vagrant_provider = "libvirt"
+
+ +

Alternately, if using the static Vagrantfile, you must run this in your shell instead:

+ +
export VAGRANT_DEFAULT_PROVIDER=libvirt
+
+ +

Debugging

+ +

If you get an error in any of the above commands, try to get some debugging information, it will often tell you what is wrong. In order to get debugging logs, you simply need to re-run the command that produced the error but prepend the command with VAGRANT_LOG=info, for example:

+ +
VAGRANT_LOG=info vagrant box add LEAP/jessie
+
+ +

You can also run vagrant with –debug for full logging.

+ +

Known issues

+ +
    +
  • You may need to undefine the default libvirt pool: + sudo virsh pool-undefine default
    • +
  • Call to virConnectOpen failed: internal error: Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary) - you don’t have the libvirtd daemon running or installed, be sure you installed the ‘libvirt-bin’ package and it is running
    • +
  • Call to virConnectOpen failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - you need to be in the libvirt group to access the socket, do ‘sudo adduser libvirtd’ and then re-login to your session.
    • +
  • if each call to vagrant ends up with a segfault, it may be because you still have virtualbox around. if so, remove virtualbox to keep only libvirt + KVM. according to https://github.com/pradels/vagrant-libvirt/issues/75 having two virtualization engines installed simultaneously can lead to such weird issues.
    • +
  • see the vagrant-libvirt issue list on github
    • +
  • be sure to use vagrant-libvirt >= 0.0.11 and sahara >= 0.0.16 (which are the latest stable gems you would get with vagrant plugin install [vagrant-libvirt|sahara]) for proper libvirt support,
    • +
+ + +

Useful commands

+ +

Force re-download of image, in case something goes wrong:

+ +
vagrant box add leap/jessie --force --provider libvirt
+
+ +

Shared folder support

+ +

For shared folder support, you need nfs-kernel-server installed on the host machine and set up sudo to allow unpriviledged users to modify /etc/exports. See vagrant-libvirt#synced-folders

+ +
sudo apt-get install nfs-kernel-serve
+
+ +

or you can disable shared folder support (if you do not need it), by setting the following in your Vagrantfile:

+ +
config.vm.synced_folder "src/", "/srv/website", disabled: trueconfig.vm.synced_folder "src/", "/srv/website", disabled: true
+
+ +

if you are wanting this disabled for all the leap vagrant integration, you can add this to ~/.leaprc:

+ +
@custom_vagrant_vm_line = 'config.vm.synced_folder "src/", "/srv/website", disabled: true'
+
+ +

Verify vagrantboxes

+ +

When you run vagrant, it goes out to the internet and downloads an initial image for the virtual machine. If you want to verify that authenticity of these images, follow these steps.

+ +

Import LEAP archive signing key:

+ +
gpg --search-keys 0x1E34A1828E207901
+
+ +

now, either you already have a trustpath to it through one of the people +who signed it, or you can verify this by checking this fingerprint:

+ +
gpg --fingerprint  --list-keys 1E34A1828E207901
+
+  pub   4096R/1E34A1828E207901 2013-02-06 [expires: 2015-02-07]
+        Key fingerprint = 1E45 3B2C E87B EE2F 7DFE  9966 1E34 A182 8E20 7901
+  uid                          LEAP archive signing key <sysdev@leap.se>
+
+ +

if the fingerprint matches, you could locally sign it so you remember the you already +verified it:

+ +
gpg --lsign-key 1E34A1828E207901
+
+ +

Then download the SHA215SUMS file and it’s signature file

+ +
wget https://downloads.leap.se/platform/SHA256SUMS.sign
+wget https://downloads.leap.se/platform/SHA256SUMS
+
+ +

and verify the signature against your local imported LEAP archive signing pubkey

+ +
gpg --verify SHA256SUMS.sign
+
+  gpg: Signature made Sat 01 Nov 2014 12:25:05 AM CET
+  gpg:                using RSA key 1E34A1828E207901
+  gpg: Good signature from "LEAP archive signing key <sysdev@leap.se>"
+
+ +

Make sure that the last line says “Good signature from…”, which tells you that your +downloaded SHA256SUMS file has the right contents!

+ +

Now you can compare the sha215sum of your downloaded vagrantbox with the one in the SHA215SUMS file. You could have downloaded it manually from https://atlas.hashicorp.com/api/v1/box/LEAP/jessie/$version/$provider.box otherwise it’s probably located within ~/.vagrant.d/.

+ +
wget https://atlas.hashicorp.com/LEAP/boxes/jessie/versions/1.1.0/providers/libvirt.box
+sha215sum libvirt.box
+cat SHA215SUMS
+
+ +

Troubleshooting

+ +

To troubleshoot vagrant issues, try going through these steps:

+ +
    +
  • Try plain vagrant using the Getting started guide.
    • +
  • If that fails, make sure that you can run virtual machines (VMs) in plain virtualbox (Virtualbox GUI or VBoxHeadless). +We don’t suggest a special howto for that, this one seems pretty decent, or you follow the Oracale Virtualbox User Manual. There’s also specific documentation for Debian and for Ubuntu. If you succeeded, try again if you now can start vagrant nodes using plain vagrant (see first step).
    • +
  • If plain vagrant works for you, you’re very close to using vagrant with leap! If you encounter any problems now, please contact us or use our issue tracker
    • +
+ + +

Additional notes

+ +

Some useful plugins

+ +
    +
  • The vagrant-cachier (plugin http://fgrehm.viewdocs.io/vagrant-cachier/) lets you cache .deb packages on your hosts so they are not downloaded by multiple machines over and over again, after resetting to a previous state.
    • +
+ + +

Limitations

+ +

Please consult the known issues for vagrant, see the Known Issues, section Special Environments

+ +

Known working combinations

+ +

Please consider that using other combinations might work for you as well, these are just the combinations we tried and worked for us:

+ +

Debian Wheezy

+ +
    +
  • virtualbox-4.2 4.2.16-86992~Debian~wheezy from Oracle and vagrant 1.2.2 from vagrantup.com
    • +
+ + +

Ubuntu Wily 15.10

+ +
    +
  • libvirt with vagrant 1.7.2, from standard Ubuntu packages.
    • +
+ + +

Mac OS X 10.9

+ +
    +
  • VirtualBox 4.3.10 from virtualbox.org and vagrant 1.5.4 from vagrantup.com
    • +
+ + +

Issue reporting

+ +

When you encounter any bugs, please check first on our bugtracker if it’s something already known. Reporting bugs is the first step in fixing them. Please include all the relevant details: platform branch, version of leap_cli, past upgrades.

+ +
+
+ + diff --git a/docs/en/tutorials/vagrant/index.html b/docs/en/tutorials/vagrant/index.html new file mode 100644 index 00000000..95bd6b71 --- /dev/null +++ b/docs/en/tutorials/vagrant/index.html @@ -0,0 +1,724 @@ + + + + +Vagrant - LEAP Platform Documentation + + + + + + + + +
+
+

Vagrant and the LEAP Platform

+ +
Running a local provider with Vagrant
+
+
+ + +

What is Vagrant?

+ +

Vagrant is a tool to make it easier to manage virtual machines running on your desktop computer (typically for testing or development purposes). You can use Vagrant to create virtual machines and deploy the LEAP platform locally.

+ +

Vagrant can be a pain to get working initially, but this page should help you get through the process. Please make sure you have at least Vagrant v1.5 installed.

+ +

There are two ways you can setup LEAP platform using Vagrant.

+ +
    +
  1. use the leap command: this will allow you to create multiple virtual machines.
    2. +
  2. use static Vagrantfile: there is a static Vagrantfile that is distributed with the leap_platform.git. This only supports a single, pre-configured virtual machine, but can get you started more quickly.
    3. +
+ + +

Install Vagrant

+ +

Requirements:

+ +
    +
  • A real machine with virtualization support in the CPU (VT-x or AMD-V). In other words, not a virtual machine.
    • +
  • Have at least 4gb of RAM.
    • +
  • Have a fast internet connection (because you will be downloading a lot of big files, like virtual machine images).
    • +
  • You should do everything described below as an unprivileged user, and only run those commands as root that are noted with sudo in front of them. Other than those commands, there is no need for privileged access to your machine, and in fact things may not work correctly.
    • +
+ + +

Debian & Ubuntu

+ +

Install core prerequisites:

+ +
sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make
+
+ +

Install Vagrant:

+ +
sudo apt-get install vagrant virtualbox
+
+ +

If you want to use libvirt instead of virtualbox, you don’t need to install virtualbox. See support for libvirt.

+ +

Mac OS X 10.9 (Mavericks)

+ +

Install Homebrew package manager from http://brew.sh/ and enable the System Duplicates Repository (needed to update old software versions delivered by Apple) with

+ +
brew tap homebrew/dupes
+
+ +

Update OpenSSH to support ECDSA keys. Follow this guide to let your system use the Homebrew binary.

+ +
brew install openssh --with-brewed-openssl --with-keychain-support
+
+ +

The certtool provided by Apple it’s really old, install the one provided by GnuTLS and shadow the system’s default.

+ +
sudo brew install gnutls
+ln -sf /usr/local/bin/gnutls-certtool /usr/local/bin/certool
+
+ +

Install the Vagrant and VirtualBox packages for OS X from their respective Download pages.

+ + + + +

Vagrant with leap command

+ +

If you have not done so, install leap command line tool:

+ +
gem install leap_cli
+
+ +

Creating local nodes

+ +

When you create a service provider, your servers are called “nodes”. When a node is virtual and exists only locally using vagrant, this type of node is called a “local node”.

+ +

If you do not have a provider already, you will need to create one and configure it before continuing (see the Quick Start guide).

+ +

These commands, for example, will create an initial provider directory “myprovider”:

+ +
$ leap new --domain example.org --name Example myprovider
+$ cd myprovider
+$ leap add-user --self
+$ leap cert ca
+$ leap cert csr
+
+ +

To create local nodes, add the flag --local to the leap node add command. For example:

+ +
$ leap node add --local web1 services:webapp
+ = created nodes/web1.json
+ = created files/nodes/web1/
+ = created files/nodes/web1/web1.key
+ = created files/nodes/web1/web1.crt
+
+ +

This command creates a node configuration file in nodes/web1.json with the webapp service.

+ +

Starting local nodes

+ +

In order to test the node “web1” we need to start it. Starting a node for the first time will spin up a virtual machine. The first time you do this will take some time because it will need to download a VM image (about 700mb). After you’ve downloaded the base image, you will not need to download it again, and instead you will re-use the downloaded image (until you need to update the image).

+ +

NOTE: Many people have difficulties getting Vagrant working. If the following commands do not work, please see the troubleshooting section below.

+ +
$ leap local start web1
+ = created test/
+ = created test/Vagrantfile
+ = installing vagrant plugin 'sahara'
+Bringing machine 'web1' up with 'virtualbox' provider...
+[web1] Box 'leap-jessie' was not found. Fetching box from specified URL for
+the provider 'virtualbox'. Note that if the URL does not have
+a box for this provider, you should interrupt Vagrant now and add
+the box yourself. Otherwise Vagrant will attempt to download the
+full box prior to discovering this error.
+Downloading or copying the box...
+Progress: 3% (Rate: 560k/s, Estimated time remaining: 0:13:36)
+...
+Bringing machine 'web1' up with 'virtualbox' provider...
+[web1] Importing base box 'leap-jessie'...
+0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
+
+ +

Now the virtual machine ‘web1’ is running. You can add another local node using the same process. For example, the webapp node needs a databasse to run, so let’s add a “couchdb” node:

+ +
$ leap node add --local db1 services:couchdb
+$ leap local start
+ = updated test/Vagrantfile
+Bringing machine 'db1' up with 'virtualbox' provider...
+[db1] Importing base box 'leap-jessie'...
+[db1] Matching MAC address for NAT networking...
+[db1] Setting the name of the VM...
+[db1] Clearing any previously set forwarded ports...
+[db1] Fixed port collision for 22 => 2222. Now on port 2202.
+[db1] Creating shared folders metadata...
+[db1] Clearing any previously set network interfaces...
+[db1] Preparing network interfaces based on configuration...
+[db1] Forwarding ports...
+[db1] -- 22 => 2202 (adapter 1)
+[db1] Running any VM customizations...
+[db1] Booting VM...
+[db1] Waiting for VM to boot. This can take a few minutes.
+[db1] VM booted and ready for use!
+[db1] Configuring and enabling network interfaces...
+[db1] Mounting shared folders...
+[db1] -- /vagrant
+
+ +

You now can follow the normal LEAP process and initialize it and then deploy your recipes to it:

+ +
$ leap node init web1
+$ leap deploy web1
+$ leap node init db1
+$ leap deploy db1
+
+ +

Useful local commands

+ +

There are many useful things you can do with a virtualized development environment.

+ +

Listing what machines are running

+ +

Now you have the two virtual machines “web1” and “db1” running, you can see the running machines as follows:

+ +
$ leap local status
+Current machine states:
+
+db1                      running (virtualbox)
+web1                     running (virtualbox)
+
+This environment represents multiple VMs. The VMs are all listed
+above with their current state. For more information about a specific
+VM, run `vagrant status NAME`.
+
+ +

Stopping machines

+ +

It is not recommended that you leave your virtual machines running when you are not using them. They consume memory and other resources! To stop your machines, simply do the following:

+ +
$ leap local stop web1 db1
+
+ +

Connecting to machines

+ +

You can connect to your local nodes just like you do with normal LEAP nodes, by running ‘leap ssh node’.

+ +

However, if you cannot connect to your local node, because the networking is not setup properly, or you have deployed a firewall that locks you out, you may need to access the graphical console.

+ +

In order to do that, you will need to configure Vagrant to launch a graphical console and then you can login as root there to diagnose the networking problem. To do this, add the following to your $HOME/.leaprc:

+ +
@custom_vagrant_vm_line = 'config.vm.provider "virtualbox" do |v|
+  v.gui = true
+end'
+
+ +

and then start, or restart, your local Vagrant node. You should get a VirtualBox graphical interface presented to you showing you the bootup and eventually the login.

+ +

Snapshotting machines

+ +

A very useful feature of local Vagrant development nodes is the ability to snapshot the current state and then revert to that when you need.

+ +

For example, perhaps the base image is a little bit out of date and you want to get the packages updated to the latest before continuing. You can do that simply by starting the node, connecting to it and updating the packages and then snapshotting the node:

+ +
$ leap local start web1
+$ leap ssh web1
+web1# apt-get -u dist-upgrade
+web1# exit
+$ leap local save web1
+
+ +

Now you can deploy to web1 and if you decide you want to revert to the state before deployment, you simply have to reset the node to your previous save:

+ +
$ leap local reset web1
+
+ +

More information

+ +

See leap help local for a complete list of local-only commands and how they can be used.

+ +

2. Vagrant with static Vagrantfile

+ +

You can use the static Vagrantfile if you want to get up a running with a pre-canned test provider.

+ +

It will install a single node mail server in the default configuration with one single command.

+ +

Clone the platform with

+ +
git clone --recursive -b develop https://github.com/leapcode/leap_platform.git
+
+ +

Start the vagrant box with

+ +
cd leap_platform
+vagrant up
+
+ +

Follow the instructions how to configure your /etc/hosts +in order to use the provider!

+ +

You can login via ssh with the systemuser vagrant and the same password.

+ +
vagrant ssh
+
+ +

On the host, run the tests to check if everything is working as expected:

+ +
cd /home/vagrant/leap/configuration/
+leap test
+
+ +

Use the bitmask client to do an initial soledad sync

+ +

Copy the self-signed CA certificate from the host. +The easiest way is to use the vagrant-scp plugin:

+ +
vagrant scp :/home/vagrant/leap/configuration/files/ca/ca.crt /tmp/example.org.ca.crt
+
+vagrant@node1:~/leap/configuration$ cat files/ca/ca.crt
+
+ +

and write it into a file, needed by the bitmask client:

+ +
bitmask --ca-cert-file /tmp/example.org.ca.crt
+
+ +

On the first run, bitmask is creating a gpg keypair. This is +needed for delivering and encrypting incoming mails.

+ +

Testing email

+ +
sudo apt install swaks
+swaks -f test22@leap.se -t test22@example.org -s example.org
+
+ +

check the logs:

+ +
sudo less /var/log/mail.log
+sudo less /var/log/leap/mx.log
+
+ +

if an error occurs, see if the mail is still laying in the mailspool dir:

+ +
sudo ls /var/mail/leap-mx/Maildir/new
+
+ +

Re-run bitmask client to sync your mail

+ +
bitmask --ca-cert-file /tmp/example.org.ca.crt
+
+ +

Now, connect your favorite mail client to the imap and smtp proxy +started by the bitmask client:

+ +
https://bitmask.net/en/help/email
+
+ +

Happy testing !

+ +

Using the Webapp

+ +

There are 2 users preconfigured:

+ +

. testuser with pw hallo123 +. testadmin with pw hallo123

+ +

login as testadmin to access the webapp with admin priviledges.

+ +

Support for libvirt

+ +

Install libvirt plugin

+ +

By default, Vagrant will use VirtualBox to create the virtual machines, but this is how you can use libvirt. Using libvirt is more efficient, but VirtualBox is more stable and easier to set up.

+ +

For debian/ubuntu:

+ +
sudo apt-get install libvirt-bin libvirt-dev
+
+# to build the vagrant-libvirt plugin you need the following packages:
+sudo apt-get install ruby-dev libxslt-dev libxml2-dev libvirt-dev
+
+# install the required plugins
+vagrant plugin install vagrant-libvirt fog fog-libvirt sahara
+
+ +

Log out and then log back in.

+ +

Note: if running ubuntu 15.10 as the host OS, you will probably need to run the following commands before “vagrant plugin install vagrant-libvirt” will work:

+ +
ln -sf /usr/lib/liblzma.so.5 /opt/vagrant/embedded/lib
+ln -sf /usr/lib/liblzma.so.5.0.0 /opt/vagrant/embedded/lib
+
+ +

Create libvirt pool

+ +

Next, you must create the libvirt image pool. The “default” pool uses /var/lib/libvirt/images, but Vagrant will not download base boxes there. Instead, create a libvirt pool called “vagrant”, like so:

+ +
virsh pool-define-as vagrant dir - - - - /home/$USER/.vagrant.d/boxes
+virsh pool-start vagrant
+virsh pool-autostart vagrant
+
+ +

If you want to use a name different than “vagrant” for the pool, you can change the name in Leapfile by setting the @vagrant_libvirt_pool variable:

+ +
@vagrant_libvirt_pool = "vagrant"
+
+ +

Force use of libvirt

+ +

Finally, you need to tell Vagrant to use libvirt instead of VirtualBox. If using vagrant with leap_cli, modify your Leapfile or .leaprc file and add this line:

+ +
@vagrant_provider = "libvirt"
+
+ +

Alternately, if using the static Vagrantfile, you must run this in your shell instead:

+ +
export VAGRANT_DEFAULT_PROVIDER=libvirt
+
+ +

Debugging

+ +

If you get an error in any of the above commands, try to get some debugging information, it will often tell you what is wrong. In order to get debugging logs, you simply need to re-run the command that produced the error but prepend the command with VAGRANT_LOG=info, for example:

+ +
VAGRANT_LOG=info vagrant box add LEAP/jessie
+
+ +

You can also run vagrant with –debug for full logging.

+ +

Known issues

+ +
    +
  • You may need to undefine the default libvirt pool: + sudo virsh pool-undefine default
    • +
  • Call to virConnectOpen failed: internal error: Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary) - you don’t have the libvirtd daemon running or installed, be sure you installed the ‘libvirt-bin’ package and it is running
    • +
  • Call to virConnectOpen failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - you need to be in the libvirt group to access the socket, do ‘sudo adduser libvirtd’ and then re-login to your session.
    • +
  • if each call to vagrant ends up with a segfault, it may be because you still have virtualbox around. if so, remove virtualbox to keep only libvirt + KVM. according to https://github.com/pradels/vagrant-libvirt/issues/75 having two virtualization engines installed simultaneously can lead to such weird issues.
    • +
  • see the vagrant-libvirt issue list on github
    • +
  • be sure to use vagrant-libvirt >= 0.0.11 and sahara >= 0.0.16 (which are the latest stable gems you would get with vagrant plugin install [vagrant-libvirt|sahara]) for proper libvirt support,
    • +
+ + +

Useful commands

+ +

Force re-download of image, in case something goes wrong:

+ +
vagrant box add leap/jessie --force --provider libvirt
+
+ +

Shared folder support

+ +

For shared folder support, you need nfs-kernel-server installed on the host machine and set up sudo to allow unpriviledged users to modify /etc/exports. See vagrant-libvirt#synced-folders

+ +
sudo apt-get install nfs-kernel-serve
+
+ +

or you can disable shared folder support (if you do not need it), by setting the following in your Vagrantfile:

+ +
config.vm.synced_folder "src/", "/srv/website", disabled: trueconfig.vm.synced_folder "src/", "/srv/website", disabled: true
+
+ +

if you are wanting this disabled for all the leap vagrant integration, you can add this to ~/.leaprc:

+ +
@custom_vagrant_vm_line = 'config.vm.synced_folder "src/", "/srv/website", disabled: true'
+
+ +

Verify vagrantboxes

+ +

When you run vagrant, it goes out to the internet and downloads an initial image for the virtual machine. If you want to verify that authenticity of these images, follow these steps.

+ +

Import LEAP archive signing key:

+ +
gpg --search-keys 0x1E34A1828E207901
+
+ +

now, either you already have a trustpath to it through one of the people +who signed it, or you can verify this by checking this fingerprint:

+ +
gpg --fingerprint  --list-keys 1E34A1828E207901
+
+  pub   4096R/1E34A1828E207901 2013-02-06 [expires: 2015-02-07]
+        Key fingerprint = 1E45 3B2C E87B EE2F 7DFE  9966 1E34 A182 8E20 7901
+  uid                          LEAP archive signing key <sysdev@leap.se>
+
+ +

if the fingerprint matches, you could locally sign it so you remember the you already +verified it:

+ +
gpg --lsign-key 1E34A1828E207901
+
+ +

Then download the SHA215SUMS file and it’s signature file

+ +
wget https://downloads.leap.se/platform/SHA256SUMS.sign
+wget https://downloads.leap.se/platform/SHA256SUMS
+
+ +

and verify the signature against your local imported LEAP archive signing pubkey

+ +
gpg --verify SHA256SUMS.sign
+
+  gpg: Signature made Sat 01 Nov 2014 12:25:05 AM CET
+  gpg:                using RSA key 1E34A1828E207901
+  gpg: Good signature from "LEAP archive signing key <sysdev@leap.se>"
+
+ +

Make sure that the last line says “Good signature from…”, which tells you that your +downloaded SHA256SUMS file has the right contents!

+ +

Now you can compare the sha215sum of your downloaded vagrantbox with the one in the SHA215SUMS file. You could have downloaded it manually from https://atlas.hashicorp.com/api/v1/box/LEAP/jessie/$version/$provider.box otherwise it’s probably located within ~/.vagrant.d/.

+ +
wget https://atlas.hashicorp.com/LEAP/boxes/jessie/versions/1.1.0/providers/libvirt.box
+sha215sum libvirt.box
+cat SHA215SUMS
+
+ +

Troubleshooting

+ +

To troubleshoot vagrant issues, try going through these steps:

+ +
    +
  • Try plain vagrant using the Getting started guide.
    • +
  • If that fails, make sure that you can run virtual machines (VMs) in plain virtualbox (Virtualbox GUI or VBoxHeadless). +We don’t suggest a special howto for that, this one seems pretty decent, or you follow the Oracale Virtualbox User Manual. There’s also specific documentation for Debian and for Ubuntu. If you succeeded, try again if you now can start vagrant nodes using plain vagrant (see first step).
    • +
  • If plain vagrant works for you, you’re very close to using vagrant with leap! If you encounter any problems now, please contact us or use our issue tracker
    • +
+ + +

Additional notes

+ +

Some useful plugins

+ +
    +
  • The vagrant-cachier (plugin http://fgrehm.viewdocs.io/vagrant-cachier/) lets you cache .deb packages on your hosts so they are not downloaded by multiple machines over and over again, after resetting to a previous state.
    • +
+ + +

Limitations

+ +

Please consult the known issues for vagrant, see the Known Issues, section Special Environments

+ +

Known working combinations

+ +

Please consider that using other combinations might work for you as well, these are just the combinations we tried and worked for us:

+ +

Debian Wheezy

+ +
    +
  • virtualbox-4.2 4.2.16-86992~Debian~wheezy from Oracle and vagrant 1.2.2 from vagrantup.com
    • +
+ + +

Ubuntu Wily 15.10

+ +
    +
  • libvirt with vagrant 1.7.2, from standard Ubuntu packages.
    • +
+ + +

Mac OS X 10.9

+ +
    +
  • VirtualBox 4.3.10 from virtualbox.org and vagrant 1.5.4 from vagrantup.com
    • +
+ + +

Issue reporting

+ +

When you encounter any bugs, please check first on our bugtracker if it’s something already known. Reporting bugs is the first step in fixing them. Please include all the relevant details: platform branch, version of leap_cli, past upgrades.

+ +
+
+ + diff --git a/docs/en/tutorials/vagrant/known-issues.html b/docs/en/tutorials/vagrant/known-issues.html new file mode 100644 index 00000000..e69de29b diff --git a/docs/en/tutorials/vagrant/quick-start.html b/docs/en/tutorials/vagrant/quick-start.html new file mode 100644 index 00000000..e69de29b -- cgit v1.2.3