Is the product Open Source?
Open Source means that the code of a programme is publicly available. Everybody can review it, search for vulnerabilities and develop it further. The opposite is “Closed Source”, so that no one but the developer – e.g. a company developing an app – can review it. Especially if a service is popular, Open Source is a real benefit. A lot of experts review the code and constantly improve it. Journalists do not have to trust a service that it really does what it claims to do – they can see in the code how the system works.
Does the product have an active user base?
Most anonymisation tools attempt to hide your identity in the pool of every other person using the anonymisation tool. For example: if you visit a website using Tor, your adversary may know that you use Tor, and that the website was accessed by someone using Tor. But your adversary does not know that it was you who did it. But if you are the only one who is using the tool, it could not have been someone else.
Is the product actively developed?
Like every other software, anonymisation tools contain bugs and vulnerabilities that adversaries might use to find out your identity. If the product is not actively developed, vulnerabilities might not get fixed.
Has the product contained a backdoor in the past?
We know of one anonymisation tool (JAP – Java Anon Proxy) that contained a backdoor in the past. If the product is open source and has an active user base, there is a good chance that a backdoor will get detected quickly.
The most secure data is the one that does not even exist. Therefore, it is important that an anonymisation tool does not store users’ data, for example in log files. Some (mostly free) VPN providers store and analyse that data to make money out of it. This can compromise anonymity.
Although the service itself may not have access to the content or metadata of users’ online activity, it might have to hand over certain information about its users. For example, metadata about communication – who sent what to whom, when and where – might be stored. Journalists and their sources should check whether a service is legally bound to cooperate with a government that they identified as a potential adversary in their threat model. This is especially important for VPN providers.