summaryrefslogtreecommitdiff
path: root/pages/docs/client/bundle-testing.md
blob: aa618897240a910ff2c0f4e9024886be855eda61 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
@nav_title = "Testing"
@title = "Guidelines for testing Bitmask"

Recommended setup
-----------------

VirtualBox (or similar) with virtual machines installed for supported OSs

For each system that you are going to test, you should do:

- Install the VM
- Restart the VM and check that the process is finished.
- Turn it off and make a snapshot named 'fresh install' or similar.

The OS should be installed with the default settings and no extra packages. However, you can choose your language, username, timezone, etc

Testing
-------

* Setup Instructions: https://bitmask.net/help

* Fresh install (stable): https://dl.bitmask.net/client/linux/stable/

* Fresh install (release candidate): https://dl.bitmask.net/client/linux/release-candidate/

* Upgrading from bundle: if you are running bundle version 0.7 or new then
  Bitmask should update automatically.

* Upgrading from package: if you have added deb.bitmask.net to your
  sources.list, then Bitmask should update automatically (make sure it is not
  commented  out).

Note: If you have a bundle version older than 0.7, please reinstall Bitmask.

Some notes on how to test: https://github.com/leapcode/bitmask_client/blob/develop/docs/testing-rcs.README

Test process
------------

- roll back the virtual machine to its *fresh install* state, to make sure that you're testing against a reproducible environment.
- download the bundle, verify signature (if apply), extract and run the app
- test the application (see next section)


Tests to do
-----------

- **check if the version number is the same as the current bundle version**
    - 'Help->About Bitmask'
    - `./bitmask --version`
- **correct installation of files to 'better protect privacy'**
    - `/etc/leap/resolv-update`
    - `/usr/share/polkit-1/actions/net.openvpn.gui.leap.policy`

    You should check that they get copied when the user says 'yes' and they don't get copied if the user says 'no'.
- **installation of tun/tap in Windows and MAC**
    TODO: explain more here

- **account creation**

    Recommended username template: test_bundleversion_os_arch, that way you avoid conflicts between test iterations.
    e.g.: 'test_036_debian7_64', 'test_036_win7_32', etc

    If you need to create extra users in order to test a bug or whatever, you can use 'test_036_ubuntu1204_32a', 'test_036_ubuntu1204_32b', etc

    In case of being a lot of users testing a version you may want to use your username instead of test, e.g.: 'johndoe_036_ubuntu1204_32'.
- **eip connection**

    You can check if the vpn is working entering to the site: http://wtfismyip.com

    or using the console:
    `shell> wget -qO- wtfismyip.com/json`
- **Soledad key generation**
- **Thunderbird configuration manually and using add-on**
- **Send and receive mail**

    You need to test communication between inside and outside users, e.g.: someuser@bitmask.net and otheruser@gmail.com

    A good thing to do is to subscribe to a mailing list that have a lot of activity.

- **Check if the account data is correctly synced.**

    After the account creation, have everything working and the app closed:
    - remove the configuration files created by the app (`~/.config/leap` in linux)
    - log in with your recently created credentials and check that everything is working and your mails are there too.


Problems report
---------------

You should to create an issue with the following information:

- OS, version, architecture, desktop environment (if relevant).
- bitmask.log file located in the root folder of the uncompressed bundle
- steps to reproduce

If you find a problem, try to reproduce and take note of the steps needed to get the same error.

Also, in some cases, a failure appears but if you run again is not there anymore (e.g.: some initialization issue), please report that too.

For more details look at [Contributing - Reporting bugs](https://github.com/leapcode/bitmask_client/blob/develop/CONTRIBUTING.rst#reporting-bugs)


Utils
-----

Download, extract and run helper script for linux:

    shell> ./download-extract-run-bitmask.sh

Script contents:

    #!/bin/bash
    HOST="https://dl.bitmask.net/client/linux/"
    VERSION="0.3.7"
    # FOLDER="Bitmask-linux32-${VERSION}"
    FOLDER="Bitmask-linux64-${VERSION}"
    FILE="${FOLDER}.tar.bz2"

    wget ${HOST}${FILE} && tar xjf ${FILE} && cd ${FOLDER} && ./bitmask