layout

By default, the <title> tag for this page is determined by the title field (in the Content group). You can override the tag value here.

metaTitle

media

bottomSections

By default, the description tag for this post is taken from the Excerpt field. You can override the tag value here.

metaDescription

By default, the "og:image" meta tag set for social sharing this post points to the Featured Image field (in the Content group). You can override that image here.

socialImage

markdown_content

If enabled, the title suffix defined in the site configuration is appended to the title tag of this page.

addTitleSuffix

To add or override any meta tag for this page, add entries to this list. Entries defined here take precedence over any other defaults.

metaTags

title

colors

date

author

excerpt

featuredImage

Divider

To read next

Traffic Monitoring REST API

### Context

Throughout the last few years I have prioritised reading as a daily habit. That has evolved from merely reading a book, to full book summaries with key insights, quotes, and my own thoughts and personal reflections on each topic. These notes represent my time and effort on this journey to try to always be my best version. But that is only possible if I act on what I learn, both from books and other sources.

Since I have already read 70+ books, it’s impossible to always be aware of all of the teachings I have already learned. That will only get worse with time because I have no intention to stop reading, and I also do my best to keep a learner mindset in all areas of life. For a while now, I have been playing with the idea of having a system to review these book notes, some of my favourite quotes from insipirational figures, and also other bits of knowledge I acquire from personal projects/studies.

By centralising all this kind of information in my Notion’s second brain, I can effortlessly access and revisit it all. But to take it a step further, with the power of Python, I can connect to Notion’s API, retrieve some of my personal notes, and seamlessly integrate them into an email. Then, automating the delivery of daily quote emails offers me not only a convenient way to get motivational quotes but also a system to periodically remind myself of my own personal notes and insights.

***

### Script explanation

First and foremost, the script begins by importing the required packages, including `smtplib` for email functionality, `datetime` for date-related operations, json for working with JSON data, random for generating random numbers, logging for error logging, and EmailMessage for creating email messages. Additionally, the notion_client package is imported to interact with the Notion API.



![](/images/Daily%20Site%20(1\).png)



Next, I defined several configuration variables to work with Notion’s API and e-mail functionalities.

Using the ‘notion_client’ package, we only need two variables to work with Notion’s API: the token from a Notion integration, and the id from the Notion page we want to work with. These are really easy to get, just follow [these instructions](https://developers.notion.com/docs/create-a-notion-integration).

To have the script working with e-mails, we need the SMTP server and port, and the address and password from the sender e-mail (for this case I used a gmail account). Then we just set a variable for the recipient’s email (my hotmail account).

Lastly, I set a file to work as the script’s log, for potential errors.

Note: I replaced some of the parts with underscores just for this blog’s sake, it is not supposed to be like this:



![](/images/Daily%20Site%20(2\).png)



The first function in this script is ***write_dict_to_file_as_json*** that writes the Notion page data (retrieved as a dictionary) to a JSON file. This function was used for easier inspection of the data during the script’s development.



![](/images/Daily%20Site%20(3\).png)



Then, there is ***read_text*** that retrieves the content of a Notion page given its ID, using the Notion API. This function returns a list of results, which includes the blocks of content in the page.



![](/images/Daily%20Site%20(4\).png)



The function ***safe_get*** allows accessing nested properties of a database given a dot-chained key. It handles potential errors such as missing keys, type errors, and index errors and returns None if any issue is encountered.



![](/images/Daily%20Site%20(5\).png)



To get random quotes, the function ***get_random_numbers*** generates a list of random numbers within a given range (1 to the total number of quotes) using the random.sample function. In this case, it selects five random numbers to represent the indices of quotes to be included in the email.



![](/images/Daily%20Site%20(6\).png)



To prepare the e-mail, the function ***e_mail_prep*** creates an EmailMessage object, sets the sender, recipient, subject, and content of the message, and returns the message object.

Then, ***send_e_mail*** uses the smtplib library to establish a connection with the SMTP server, login to the sender's email account, and send the email message.



![](/images/Daily%20Site%20(7\).png)



Finally, the ***main*** function serves as the entry point for the script. It encapsulates the main logic of the script, and performs the following steps:

1.  The function establishes a connection to the Notion API using the integration token and retrieves information about the specified Notion database.

2.  It queries the database to obtain the rows (quotes) stored within it.

3.  For each row, it retrieves the content, author, origin, and date properties and stores them in a simplified form as a dictionary.

4.  The function then selects five random numbers using the ***get_random_numbers*** function.

5.  It constructs the content of the email by retrieving the appropriate quote information based on the selected random numbers.

6.  An introductory and concluding message is added to the email content.

7.  The email message is prepared using the ***e_mail_prep*** `function and sent using the` ***send_e_mail*** function.

8.  If any exception occurs during the execution, an error notification email is prepared and sent to the specified recipient email address.

I purposely left 3 commented lines, which I used to inspect the data while I was developing the script, just in case someone might want to do it as well.

To end, the ***if name == 'main'***  `block ensures that the` ***main*** function is executed when the script is directly run and not when it is imported as a module.



![](/images/Daily%20Site%20(8\).png)



***

### Full Python script

Here you can see the full script, but you can also get the it from my [GitHub page](https://github.com/MPCaloba/Daily_Quotes).



![](/images/Daily%20Site%20(9\).png)



***

### Executing the script

To automate the process of sending daily quote emails, I used Windows Task Scheduler to schedule the script's execution. This tool provides a user-friendly interface to schedule and manage tasks, making it an excellent tool for automating this processes on a Windows system, and eliminates the need for manual intervention. Here are the steps to get it up and running:

1.  **Set up the Python script —** To get started, make sure you have the necessary Python script that retrieves quotes from Notion and sends them via email.

2.  **Open Windows Task Scheduler —** Open the Windows Task Scheduler application by searching for "Task Scheduler" in the Start menu.

3.  **Create a new task —** In the Task Scheduler window, click on "Create Basic Task" on the right-hand side. This will open a wizard that will guide you through the process of creating a new task.

4.  **Provide task details —** In the task creation wizard, provide a name and description for the task. For example, you can name it "Daily Quote Email" and provide a suitable description.

5.  **Choose the trigger —** Select the trigger "Daily" to specify that the task should run every day. Set the desired time for the script to execute.

6.  **Choose the action —** Select the action "Start a program" to specify the program that will be executed. Click "Next" to proceed.

7.  **Configure the Action —** In the "Program/script" field, provide the path to your Python interpreter. This is usually located in the Python installation directory, such as ***C:\Python\python.exe***, but you can find it by opening the command prompt and running ***where python***. In the "Add arguments" field, provide the path to your actual Python script.

8.  **Finalise and save the task —** Review the task details on the summary screen and click "Finish" to create the task. The task scheduler will now execute the Python script automatically at the specified time every day.

***

### Resources

*   [Resource 1](https://www.python-engineer.com/posts/notion-api-python/) and [Resource 2](https://www.youtube.com/watch?v=M1gu9MDucMA&ab_channel=PatrickLoeber): These links (both the article and the video) serve as practical guides for working with the Notion API in Python, covering the setup process, creating, querying, updating, and deleting pages in a Notion database.

*   [Resource 3](https://www.youtube.com/playlist?list=PLS_o2ayVCKvDwzhB-wdzBTvpQwgArrHlY): This is great playlist (with just 5 minute videos) to understand the basics of reading and writing data through Notion’s API, both in pages and databases.

*   [Resource 4](https://developers.notion.com/reference/intro): For more informations about Notion’s API, you can also check the official documention available.

***

### Links to other platforms

[LinkdIn](https://www.linkedin.com/in/marcocaloba/); [GitHub](https://github.com/MPCaloba); [Medium](https://medium.com/@marco_caloba); [Goodreads](https://www.goodreads.com/user/show/79502560-marco-caloba).


Daily Quotes

### Context

In today's fast-paced world, personal development is a key focus for many individuals seeking growth and self-improvement. I currently do my best to read approximately 2 books per month, predominantly in the genre of personal development. To ensure that I extract the maximum value from each book, I began compiling book summaries in Notion. These summaries include all my highlights from the readings, accompanied by a more comprehensive 'book overview template' I adapted from youtuber Ali Abdaal.

For over a year, I relied on a website to access my Kobo highlights, manually copying and pasting them into Notion. The idea of automating this process had been lingering in my mind, but it wasn't until the website announced its upcoming closure that I finally took action.

Determined to streamline my note-taking procedure and integrate it seamlessly with Notion, I set out on a quest to automate the process. My search for solutions led me through many possibilities, but it wasn't until I stumbled upon an intriguing [blog post](https://www.juliariec.com/blog/export-kobo-to-notion/) that I found a promising starting point. Although her script was implemented in Node.js, which I have limited knowledge of, it inspired me to create my own solution.

Within Julia's blog post, she referenced a repository on GitHub named "[Export Kobo](https://github.com/pettarin/export-kobo)," housing a Python script capable of extracting data from the KoboReader.sqlite file and exporting it in various formats. Initially, I considered using this script as it was already an upgrade from my current “system”. However, my goal was to establish a direct connection with Notion, so I got compelled to develop my own script. Initially, I attempted to modify the existing Python script, but the complex organization and structure proved time-consuming to comprehend. Eventually, I decided to start from scratch.

Automation has revolutionised various aspects of our lives, and in my pursuit of personal development, I sought to apply it to my reading process. By developing a Python script, I was able to seamlessly integrate my Kobo highlights with Notion, saving time and effort while preserving the essence of my reading experience.

In the following sections, I will provide a comprehensive breakdown of the script's structure. I researched the subject by diving into various articles and YouTube videos. At the end, I will provide a curated list of the primary resources that greatly influenced the development of this script. It's worth noting that the process involved a significant amount of trial and error, and while I aimed to capture all valuable resources, it's possible that some may have been forgotten.

***

### Script explanation

First and foremost, we need to import the necessary packages:



![](/images/KTN%20Website%20Code%20(1\).png)



Since I might change the script’s location later on, I start by getting the folder and file directories. The name “KoboReader.sqlite” will always be the same because I will simply copy and paste it from the Kobo folder, directly to the folder with this script.



![](/images/KTN%20Website%20Code%20(2\).png)



Using the ‘notion_client’ package, we only need two variables to work with Notion’s API: the token from a Notion integration, and the id from the Notion page we want to work with. These are really easy to get, just follow [these instructions](https://developers.notion.com/docs/create-a-notion-integration).
Note: I replaced some of the letters/numbers with underscores just for this blog’s sake, it is not supposed to be like this:



![](/images/KTN%20Website%20Code%20(3\).png)



Since we want to read a file that contains a database, we need to create a connection to read the information stored in it. For this, I created a function that takes the file as an input and then opens the connection with the sqlite3 package.



![](/images/KTN%20Website%20Code%20(4\).png)



To actually write the text in a Notion page, you need to append children blocks (you can find more information about this in [Notion’s documentation](https://developers.notion.com/reference/patch-block-children)). The function bellow takes 4 inputs: the client (activated through the Notion token, by the notion_cliente package, in the main function), the page id, the text I want to write, and the type of block I want to create (you can find the list of different blocks [here](https://developers.notion.com/reference/block)). With this information, we can append a children to our page with the format presented below. I found [this video](https://www.youtube.com/watch?v=KENSTonsiEc&list=PLS_o2ayVCKvDwzhB-wdzBTvpQwgArrHlY&index=2&ab_channel=IndyDevDan) to be very helpful in understanding this part.



![](/images/KTN%20Website%20Code%20(5\).png)



On to the main function:

1.  The first step is to activate the connection to the SQLite file with the function presented above.

2.  Next, I created two dataframes:

    a.  Through an inner join between the ‘content’ and ‘Bookmark’ tables, the ‘books_in_file’ dataframe stores the name, author and last read date of the books that have highlights.

    b.  From the ‘Bookmark’ table, the ‘bookmark_df’ dataframe stores all highlights, annotations and the date they were created on, ordering from oldest to newest.

3.  Since there are some highlights that have some unwanted white spaces in the beginning and/or end, I use a *for cycle* to strip all highlights and keep only clean highlights.

4.  Then, I set the ‘client’ variable with the Notion connection to use it in the final cycle.

5.  To end this main function, I have 2 *for cycles* with 2 *if statements* to get all highlights. Here are the steps:

    a.  We start by running through all of the books we saved (the ones that have highlights) and write in the notion page a ‘heading 2’ (mid-size title) in the format ‘Book Title - Author’;

    b.  Next, we set a second cycle that, for each book saved in the previous cycle, runs through all of it’s highlights;

    c.  Then, for each highlight, we set the first condition that checks if the book title we just wrote (in the first step) is in the current highlight’s id.

    d.  Lastly, if the previous condition is true, we set a second condition to check if we have a annotation. If it is just a highlight, it writes in Notion a normal ‘paragraph’ with the content. If there is a annotation, first it writes the annotation in ‘quote’ type and then writes the highlight as a ‘paragraph’.

    e.  At the end of each book, I also write a blanck paragraph, just to create a visual separation between books.

6.  Just one last step: to run the main function, I wrote a starting block at the end.



![](/images/KTN%20Website%20Code%20(6\).png)



***

### Full Python script

Here you can see the full script, but you can also get the it from my [GitHub page](https://github.com/MPCaloba/Kobo_to_Notion/tree/main).



![](/images/KTN%20Website%20Code%20(7\).png)



***

### Final product

I might consider some changes in the future but, as of today, these are the steps I need to take to get my highlights in my Notion database:

1.  I connect my Kobo to my laptop and copy the ‘KoboReader.sqlite’ file to the folder where I have the Python script stored.

2.  I run the script in VSCode.

3.  I open my books Notion database and copy the notes from the page I used in the script (the first block in the image below, ‘Kobo to Notion’), to the actual page of the book summary.



![](/images/Notion%20Database.PNG)



***

### Resources

*   [Resource 1](https://www.juliariec.com/blog/export-kobo-to-notion/): This was the first blog that caught my attention when I was searching for ways to connect my Kobo device with Notion.

*   [Resource 2](https://github.com/pettarin/export-kobo): This repository was mentioned the above article, and it is a Python script capable of reading the KoboReader.sqlite file and exporting the data in several formats.

*   [Resource 3](https://www.youtube.com/playlist?list=PLS_o2ayVCKvDwzhB-wdzBTvpQwgArrHlY): This is great playlist (with just 5 minute videos) to understand the basics of reading and writing data through Notion’s API, both in pages and databases.

*   [Resource 4](https://www.sqlitetutorial.net/sqlite-python/sqlite-python-select/): This article was very helpful to understand how I could create a connection to the database with a Python script.

*   [Resource 5](https://inloop.github.io/sqlite-viewer/): I used this website to upload my ‘KoboReader.sqlite’ file, so I could understand how the tables were organised and where was the information I needed.

*   [Resource 6](https://developers.notion.com/reference/intro): For more informations about Notion’s API, you can also check the official documention available.

***

### Links to other platforms

[LinkdIn](https://www.linkedin.com/in/marcocaloba/); [GitHub](https://github.com/MPCaloba); [Medium](https://medium.com/@marco_caloba); [Goodreads](https://www.goodreads.com/user/show/79502560-marco-caloba).


Kobo to Notion

### Context

This is a Python program that reads an excel file with information from eletrophysiology recordings on brain slices, analyses it and exports the required parameters in an accessible way. I made it for my girlfriend, who is currently a Neurocience Masters’ student. She is studying the role of inflammation on an epilepsy model, using hippocampal slices to assess both molecular and electrophysiology hallmarks in the progression of the disease. She submits those slices to a 40-minute protocol where a specific program detects spontaneous brain activity and then returns those oscillations in data: time and amplitude. She asked for my help to optimise that data’s treatment because the method she used was archaic and very time consuming. From the results the program returns, she needed to apply certain temporal conditions to find bursts of activity, on top of which she needed to extract the number of bursts, the duration and frequency of each burst, the number of events per burst and the peak and average amplitude of each burst. All of this for a single sample, and she has around 12 samples per week.

Please keep in mind this was my first time trying to create something like this so I am positive that there are more efficient ways to simulate what I did. I still have a lot to develop in my Python skills, but the goal with this program was to provide, with what I know today, an user-friendly solution that could simplify my girlfriend's tasks of analysing each sample's data.

You can also find this blog post in my Medium page, and the script and some sample datasets in my [GitHub](https://github.com/MPCaloba/EletroPy).

***

### The program explained

The first part of the code is just to import the required libraries:



![](/images/Website%20Code%20(1\).png)



Then, to start it off, first we get the current folder’s directory so in the end we can just have the executable in the folder where we have our files. Since there might be more than one file to be analysed, I wanted to make a script that could handle multiple analyses so it would be easier and quicker to use.



![](/images/Website%20Code%20(2\).png)



Since we might be working with multiple files, I decided to create an empty dictionary so we could store in there, separately, the data for each file. I also created a variable for the file number but it is only used to increment at the end of each for loop, so it is always reading and analysing the correct file. From here on, everything will be inside a for loop that runs through all files in the folder.



![](/images/Website%20Code%20(3\).png)



Here we will start the analysis that is made to each file. First, we store the name of the file and then, in the dictionary, read the excel file to a dataframe. After that we clear all rows with empty values. These are mostly at the beginning and end of the file, where some information is missing.



![](/images/Website%20Code%20(4\).png)



We can now get the number of entries each file has and calculate the frequency for each row. To do so, we considered that the period of each wave starts with the start time of one row and ends with the start time of the next row.



![](/images/Website%20Code%20(5\).png)



The whole goal of this analysis is to see if each file’s sample had a “burst” during the experiment. A burst is considered to be an event that lasts at least 6 seconds and, during that period, all activity has frequency higher than 2 Hz and all occurrences are less than 2 second apart. So now we establish the last two conditions mentioned, for a first analysis.



![](/images/Website%20Code%20(6\).png)



With these conditions, we now apply them to the dataset and create a new column where 1 means that the row fulfills both conditions, and 0 means that at least one condition is not met.



![](/images/Website%20Code%20(7\).png)



Theoretically, the data should both start and end with a 0 in the column we created previous to this. But I found that there are some cases where the file starts or ends with a row that meets the conditions and these didn’t work well with the method I used to apply the last condition to detect bursts. To prevent these issues (probably related with data inconsistencies), I created two extra rows, one at the beginning and one at the end, with the same data as the rows next to them but then I force the column ‘Possible Burst’ to have the value 0.



![](/images/Website%20Code%20(8\).png)



We now know where there are possible bursts, we just need to apply the third condition. To do so, we create an empty dataframe to calculate the period between each 0 to 1 and 1 to 0 change. With this, we can then select only the events that lasted more than 6 seconds as the last condition.



![](/images/Website%20Code%20(9\).png)



If there happens to be a file that does not include a burst, there is no need to continue analysing it’s data. In this step, if that is the case, we create a text file with the name of the sample just saying that there are no bursts in this dataset.



![](/images/Website%20Code%20(10\).png)



For the files that do actually have bursts, we first add a column to know how many bursts there are. After that, we need to get the actual data from the interval of each burst. Since the previous information just had the times at which it started and ended, we get the indexes from those rows and then use those indexes to create a new dataframe we all of the data from all of the bursts.



![](/images/Website%20Code%20(11\).png)



Aside from calculating some outputs with the data from all bursts, we also want information on each burst, individually. To get that, we create an empty dictionary and store there each burst’s information.



![](/images/Website%20Code%20(12\).png)



On to the results! First, for all bursts’ data, we get the number of events and number of bursts, and then some averages per burst: number of events, burst time, peak amplitude and frequency. All of these are stored in a dataframe to easily export the data.



![](/images/Website%20Code%20(13\).png)



For each burst, we get the number of events, burst time, average peak amplitude and average frequency.



![](/images/Website%20Code%20(14\).png)



Before exporting the results, in the folder we are working, we create a new folder with the name of the sample and store it’s directory.



![](/images/Website%20Code%20(15\).png)



To end the program, we export the results in a new excel file and save it to the new folder. After that, we send to that same folder the original file, used for the analysis, and then increment the number of the file.



![](/images/Website%20Code%20(16\).png)



***

### Execution of the program

Since the goal was to have an easy, user-friendly, solution, I didn’t her to rely on an IDE to run the script so I decided to create an executable from this python script. To do this I had to install the PyInstaller Package (`pip install pyinstaller`), open the cmd in the folder where I had the script (**tip:** open the actual folder and then just write ‘cmd’ in the folder’s directory) and then write the following:

![](/images/Website%20Executable-7be7f974.png)

“Program.py” is the name of the script and the “--windowed” part is so that the cmd does not open when we execute the file. Here is a quick video showing how this program works:

***

### Full python script

![](/images/Website%20Full%20Script.png)

***

### Final thoughts

This was a great test of my Python knowledge where learned a lot of features I wasn’t aware of and got even more excited about the possibilites about learning this programming language. It was also very satisfying to able to use Python in a real-case scenario where it helps someones’ actual work.

Links: [LinkdIn](https://www.linkedin.com/in/marcocaloba/); [GitHub](https://github.com/MPCaloba); [Medium](https://medium.com/@marco_caloba); [Goodreads](https://www.goodreads.com/user/show/79502560-marco-caloba).


Projects

Traffic Monitoring REST API

Daily Quotes

Kobo to Notion

ElectroPy

Google Data Analytics Capstone Project