Email and file tracking made easy

Email / File tracking explained — by freepik, freepik, amahce and dooder

Cover photo (Email / File tracking explained) — by freepikfreepikamahce and dooder.

Almost everything today is tracked in some way or another. You go for a run in the lovely weather! You phone tracks your movement. You spend some hard-earned cash in the store! The bank tracks your money movement. You watch two three shows on Netflix! Netflix tracks and record what you enjoy, to deliver better options. And who has not visited an online store, placed something in the shopping cart but changed the mind in the last second. Only to be hunted for weeks later with ads for that very same product.

With all these tracking applications running nonstop, why should you be unable to track your own information? This brings us into the theme for this article: Email and File tracking. What is it? And how can it be utilized by you? I will in this article try to first explain the legal questions behind it, then how does it work? Before ending it with how you can create your own email or file tracking server to test it out! Does it sound unexciting? 

Is it legal?

Foremost you may ask, is this legal? Well, it is a challenging question but straightforward answer yes. You can track emails just as any other messages to see when it opens and is manipulated. Think of it, doesn’t almost every chat program today of some sorts offer a method to inform you when a message is received or read? Take messenger as an example, every time you send a message you receive a little notification notifying you that it is delivered, and the other person have glimpsed it. You even have this feature on your phone when send SMS. Tracking conversations have been a natural thing ever since we started talking over a vast distance.

On the other side what can regulate how you track a message is the information you gather, and how you store it. Companies maintain strict regulations describing for them how to save data, what kind of data, and how to make your data accessible for you. This is so you at all time can track your own digital footsteps. Laws like GPDR are there for your safety.

GPDR as an example do not say email tracking is illegal. (EU, 2021) But it requires the company can show that the recipient of such an email have consented to the monitoring of their behavior. This, on the other hand, is easy to get through privacy policies and so on. How often have you read everything written in that huge document?

Illustration of information gathering
Illustration of information gathering — icons made by Freepik from www.flaticon.com

As a private person if everything you produce, is for purely personal or household activity do not GPDR apply. You are then capable to in this “grey” area to track what you desire. (GDPRHub, 2021). The reason for why email tracking is scarier than let us say SMS tracking, are most likely because of all the information you can get. All from the IP of the computer to what kind of computer, what links are clicked, when it is clicked and so much more. But how does all this work?

How does it work?

There are many ways to track emails and files but the most familiar one and easy to use are pixel tracking. It is the use of an image that’s not bigger then 1×1 pixel and often transparent. When your browser or Microsoft Word for that matter, loads an image from a URL. The site behind the URL can collect a lot of information regarding the origin of the URL request. Its not just in email trackers this is used but companies such as Facebook, Google ads and so on. They all exploit the same technology to track your movement through the web. (Facebook , 2021)

Regardless, let us focus on the mail tracking procedure. When you open an email, most programs do automatically download every image, or link that is inside the mail. It is when this happens the magic take place. The email program wants an image, and the website generates it, but at the same time it saves every single data goodie it can get it hand on.

Wait, what? Every single data you say. No no no, I have VPN, virus protection and use anonymous browsing. Well, this will not in fact support you so much. If you run VPN, Sure, they will not get your real IP at least. But still email trackers can log, when you open the mail, and what kind of device you use. Click here to see a small example of what your browser carries. The data indicated can be retrieved without your permission. Do not worry the site do not log any form of data or track it.

This is fundamental data that can be retrieved from any device. It may look frightening on its own but can be even scarier when collected and analyzed. Imagen whenever you open an email, or just scroll past it on outlook a message is sent to a server logging your behavior. This can be some powerful data for a company wanting to examine how the employer is using the work time.

Breaking it down

Let us analyze the script you just clicked on. What does it achieve? First step is to get the IP. Using php its amazingly simple. The principle we utilize is $_SERVER. It is a PHP super global variable, which contains information about, header, paths, and script location with more for the computer that are trawling the web. (W3Schools, 2021) In this example your phone or computer that you are on right now.

Therefore, by exploiting this, we can send a request with php to obtain the remote address. Remote address will be your IP because its remote from the server the PHP script are running on.

<?php 
    $ip = $_SERVER['REMOTE_ADDR'];
    echo($ip);
?>

//Output: 127.0.0.1

This will give you the IP of the one visiting your script. We can, in addition, use the IP to track the user down to which city they are sitting in when reading the email. But we require more data. What about the time? Easy we employ the same principal. But instead of asking for the address we ask for the time.

<?php 
	$time = date('Y-m-d H:i:s', $_SERVER['REQUEST_TIME']); 
	echo($time);
?>

//Output: 2021-01-08 16:34:39

Great! Now we in addition possess the time for when someone visited the script. Last thing we require that can be useful are the User-Agent.

<?php 
	$agent = $_SERVER['HTTP_USER_AGENT'];
	echo($agent);
?>

//Output: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36

These three things combined deliver us more than enough information to identify when, where, what and how. Gather enough of it and you can start to reveal patterns. Like where the user reads most of the emails. Where the user lives. Where the user work. When they read emails. What kind of devices is used to what time? It can be quite some substantial information when it is all collected. And this is as stated before, maybe one of the reasons people think email tracking are scarier than tracking a simple SMS.

Information stealing - Background vector created by dooder
Information stealing illustration — by dooder

You can collect data what now?

Great so we currently know how to collect data with php. But how can we relate this to tracking? Users still need to click on a link to visit the webpage. Or do they? This is where the magic of pixel tracking comes in handy. When visiting a webpage, downloading an email, or opening a document. Objects like images are downloaded by them self. Some programs like outlook do not perform this by default, and it tells you this is for your personal safety. Because when you download the image, all the data from before are stored somewhere in a database.

Usually, when an image is not downloaded, you will see a square of some sort telling you there is supposed to be an image there. And often will it have some sort of text providing you an idea of what the picture looks like. When tracking you do not want this, and you do not want the user to know there is an image inside the code.

Illustration of image not download in outlook.
Illustration of image not download in outlook.

So, to make the impact as little as possible we use 1×1 pixel image, and if it is loaded it is transparent. All you require for this to work is to add some small extra lines to your php script as an example:

<?php
	header("Content-Type: image/png");
	$sign = imagecreatefrompng("pixel.png");

	// Transparent
	imagealphablending($sign, false);
	imagesavealpha($sign, true);
	
	// Display the image
	imagepng($sign);
    	imagedestroy($sign);
?>

What happens here is that we set the header for the page to represent an image. So, when another program tries to access the page, it will think it is an image. At that moment we load an image here “pixel.png”, into the site, make it transparent and display it to the program that try to access. For this to work, you need creating a small image 1×1 that is transparent and save it as pixel.png.

When it all comes together

Place this all together and save it as index.php upload it to a server, and there you go. If all is done correct, this is a supersimple way to track someone.

<?php
	//Get ip
	$ip = $_SERVER['REMOTE_ADDR'];

	//Get time
	$time = date('Y-m-d H:i:s', $_SERVER['REQUEST_TIME']);

	//Get User-Agent
	$agent = $_SERVER['HTTP_USER_AGENT'];
	
	//Set mail to send to
	$to = "my@mail.com";

	//Set message
	$message = "IP: ".$ip."\nDate :".$time."\nAgent: ".$agent;

	//Send mail
	mail($to, "TRACKER CLICK", $message);
	
	//Set header and content
	header("Content-Type: image/png");
	$sign = imagecreatefrompng("an.png");

	// Transparent
	imagealphablending($sign, false);
	imagesavealpha($sign, true);

	// Display the image
	imagepng($sign);
	imagedestroy($sign);

?>

In this example script, each time someone visits the page you will receive an email with the information from the user. Next step is to force someone to visit the link unconscious. The way of achieving this is with an image tag.

<img src="https://my-site.com/folder_to_tracker/" width="1" height="1" style="display:none!important;">

The application that are reading the HTML code will look up the URL for an image, your script will provide this but also log the information. The image will be transparent, so it will not show and if loaded. The HTML code equally tells the application not to reveal if it is not downloaded correct. By making use of HTML code, you can paste it into word documents, email and much more. And each time someone opens the word document, email, or any other document, you receive a notification. It is a quite simple but a wonderfully effective way, of tracking the events of a file or email.

True this is not a particularly effective way of collecting lots of data. You need checking each and separate email, store them, obtain relevant data and so on. In the effort of making, it simple for you to get started, I can recommend TRAWO.

Create your own tracker!

TRAWO is an opensource email and file tracking service so you can continue develop it your own, use it as an example or use it as it is. TRAWO is divide into two sections, GUI and core. The core work in the backend and are the code, that tracks your files and emails. The GUI is an easy-to-use user interface for you to maintain track of all the data. You can download the source code for TRAWO here. And the GUI here.

The readme file on GitHub provides a informative introduction in how to setup TRAWO. But let us examine it anyway. First, before you jump into action, you need an online server. As mention before you need an active webserver that can provide an image and still copy the data from the user. The most convenient way of achieving this is to setup your own or buy a domain from a provider. The difference between these two options is the cost and time.

Setup a server and core script

I will not provide any link to sites selling domain names and server, solely make a quick google search and you should be able to find some reasonable ones. But there are many great tutorials for producing one by yourself. If you desire to setup your own here are some link deepening on your operation system. For TRAWO to work you need PHP, Apache and MySQL.

Remember making your server public for the whole internet, and instead of using a domain name (example.com) you will be using your IP address (eks: 127.0.0.1). Everything will work great local because you are on the same network, but if someone outside your network tries accessing your local IP, they will be unable to find you server. Remember you will have two sets of IP one local, and one wide.

Illustration of WAN and LAN
Illustration of WAN and LAN — by freepik and jemastock

Installing the core

When you have a server ready, let us start the installation of TRAWO. Start by downloading the project from gitHub. When its download unzip it and drag the CORE folder to your WWW root folder. This can be discovered in various places depending on your setup. Usually, on Linux systems is it: /var/www/html/. On windows C:\www or C:\xampp\htdocs if you for example use xampp server.

When that is done visit http://your-adress/setup.php, to launch the setup page for TRAWO. When there enter first the information for your database. The username you have gotten during install and password (this is so we are capable to login). Next enter the name of the database you want the data to be stored into.

On a MySQL server you can provide many separate databases, so we need identifying which one on the server to use. We also need knowing the address for the server, and this is usually the same as your-adress in http://your-adress/setup.php. And finally, we need to set a prefix, standard use is tracker_. This is so you can undoubtedly know which tables in the database belongs to TRAWO and make it easier to store data from different servers inside alonge with TRAWO.

Overview of install page for TRAWO core.
Overview of install page for TRAWO core.

When you are ready hit create database. This will produce all the tables and save a db_config.php file for you that contain the data you entered. If all went well, you should see DB created complete in output. You are then ready to create some users. Fill inn all the information required and click create user. You will receive a token in the output. Save it and keep it safe. This is your username and password when using TRAWO. If you manage to lose it, can it be uncovered if you login to the mySQL database and look up the user table. When all the users are created, delete the setup.php file. You do not require it anymore, and if you do not obliterate it other can acquire your information by opening the same file.

Send your first mail with a tracker

TRAWO comes with a premade GUI (Graphical user interface) that you can use to create, delete and mange trackers and contact. If you do not want to utilize it, you can create your own based on the core script. There is a thoroughly documented use of it here scroll all the way down. But for this article we are exploiting the premade one. Start by visiting google play to download the TRAWO GUI. There are three phases with 3 steps in each, we need to address to get you up and running. Phase 1 is Connect GUI, Phase 2 produce your first tracker, Phase 3 view the information.

Phase 1 - Connecting GUI to core step 1 to 3
Phase 1 – Connecting GUI to core step 1 to 3

Phase 1 – Connect GUI

You need a stable internet connection to make the app work. If you launch the app and are missing, WIFI are you asked to correct the problem. When that is completed the TRAWO GUI check if you have entered a valid token and URL. If you are lacking one of them, ether the URL or TOKEN. You get a popup requesting you to fix it.

On the settings page enter the token you got earlier while installing the TRAWO core script. And you also need entering complete URL to the core. You need entering both the domain name/ip and folder name for it to work (http://my-domain.com/core-folder). If everything works, the GUI will display a congratz message informing you to enjoy the program. If there is something wrong, it informs you to examine the settings.

Make sure you have correct IP / domain-name and folder, and that the token is correct. Tripel check also that you can contact your server from outside your own network. Try to contact it with your phone and disconnect it from your Wi-Fi to really be sure you can contact your server. And that is it. You have completed the first phase. Let us look at the next one.

Phase 2 - Creating tracker. Step 4 to 6.
Phase 2 – Creating tracker. Step 4 to 6.

Phase 2 – Create your tracker and send it

You are now prepared to produce your first tracker. If you try to enter the tracker list, and you have completed the setup and do not possess any tracker. No worries, you will get a popup informing you to generate one.

On the tracker page fill inn:

  • Mail to the person to track
    • This works as an id for the person. Therefore, you require it even if you are tracking only documents.
  • Subject
    • This will show in the tracker list and will also be auto filled into the mail if you create the mail from the GUI.
  • And what kind of tracker you are creating.
    • This is just so you organize it for yourself. It does not do anything with the tracker. It works the same if you choose, mail, doc, pdf and so on.  In future development will it change something about the tracker, but for now is it only for categorizing.  

When all this is done, you get the tracking-id below. And you also get a html-code. You can copy-paste the code into your email program. Or enter just the URL into word document as an image reference, you will then be able to see when people open the word document or read the email. You can also just click generate email, and all the information gets pasted into the mail app.

NB! Gmail is not supported if you want to use Generate email I recommend that you use outlook. This is because there is a problem when you want to insert IMG links to Gmail. For the moment thats not supported.

When you have sent the tracking mail. Sit back and relax, everything from here on goes automatically.

Phase 3 - View information. Step 7 to 9.
Phase 3 – View information. Step 7 to 9.

Phase 3 – View information

If you visit the tracker list, you can see there instantly is a tracker present now after we generated it. When you get a click, download, open or what to call it on the link, will you see the event counter goes up.

Trackers communicate information about when it was generated, who it is to, the subject and of course a counter describing how many times it has been opened. If you click on it, you obtain a list with all the events regarding the tracker. The IP with a quick lookup, time and date and user agent that was used. If you at some point do not require the information anymore just swipe left and hit delete. This work also other places in the app. And that is it, you have created your first tracker congratulation.

Manually inserting the code

Finally, some narrow tips and tricks. All the steps so far have worked great on a phone, but how can you utilize it if you sit on a computer? First of all you need a tracking ID. If you are using TRAWO GUI, go to create a new tracker and copy the HTML code provided. You can also generate it by visiting your core folder in a browser. Type in the following:

http://{YOUR-ADDR}/core/?token={YOUR-TOKEN}&addr={EMAIL-TO}&subject={YOUR-SUBJECT}

Replace all {} with your own data. The result should look like this:

{"tracker":"{TRACKER-ID}"}

Where {TRACKER-ID} is your tracker id. Copy this and add it into the following HTML code. Replacing {} with your own data:

<img src="{YOUR-URL}/core/?image={TRACKER-ID}" width="1" height="1" style="display:none!important;">

What you currently possess is a HTML code ready for being used in various settings. All you need to do is to copy past it into a document, object or file that can read HTML. Two examples for this are pasting it into outlook, or a word document. Let us have a look at them.

Insert HTML Source code Outlook emails

First create a new text document, and insert the HTML code from before. Rename your text document from New Textdocument.txt to my-html-code.html. Great you are now prepared.  

1. Open a new email in outlook.

Illustration of New Email Outlook
Illustration of New Email Outlook

2. Select more commands to customize your quick access tool.

Illustration of Edit Toolbar in Outlook
Illustration of Edit Toolbar in Outlook

3. Choose “attach file” and “add” it to your toolbar and click ok.

Illustration of Customize Toolbar in Outlook
Illustration of Customize Toolbar in Outlook

4. Open the “attach file” window from you quick access toolbar.

Illustration of add attachment in Outlook
Illustration of add attachment in Outlook

5. Select your HTML file, do NOT click insert, but click the little arrow on the side and choose Insert as Text.

Illustration of insert html code in Outlook
Illustration of insert html code in Outlook

And that is it! Your HTML code are now loaded into your Outlook email and you are ready to send the email.

Insert tracking code to word

All you need here is the URL to your tracker.

1. Go to Insert, and find Quick Parts.

Illustration of insert and quick parts in word
Illustration of insert and quick parts in word

2. Click on Field.

Illustration of Field in word
Illustration of Field in word

3. Choose Include Picture, enter your URL and click on Data not stored with Document, and click on OK.

Illustration of includePicture in word
Illustration of includePicture in word

That’s it, and what you currently have achieved is inserting an image to word that will load each time word opens on another computer. The image is unstored into word, so it needs to be downloaded each time. 

Summary

My first goal of this article was to try explaining the legal question behind tracking. A quick look back will tell us that it is legal for both companies and private people. But companies require an accept from the users, private person are unrequired by law to do this if it is for purely personal or household activity. But well yes, this is a grey area.

Next was how trackers work. The straightforward answer to this is there are many ways, depending on the program or script. But all have in common that they make use of another service such an image to conceal them self while logging activity. The most common used tracker method is pixel tracking, where you display a 1×1 pixel image after collecting the data.

At long last, we tried creating our own tracking server using TRAWO. I hope you had fun reading, and that it was instructive. The whole article has been written considerably basic, all in the effort to make it easy to understand, even for those who do not use computers for a living.

Reference

EU, G. (2021, 01 08). GDPR. Collected from Email – Tracking: https://www.gdpreu.org/compliance/email-tracking/

Facebook . (2021, 01 08). Facebook for Business. Collected from Create and install a Facebook pixel: https://www.facebook.com/business/help/952192354843755?id=1205376682832142

GDPRHub. (2021, 01 08). GDPRHub. Collected from Article 2 GDPR: https://gdprhub.eu/Article_2_GDPR

W3Schools. (2021, 01 09). W3Schools. Collected from PHP Superglobal – $_SERVER: https://www.w3schools.com/php/php_superglobals_server.asp

Legg inn en kommentar