by Victor Matekole

Skip to content

Personal Data Warehouses

Posted on:14 December 2023 at 06:56

In the previous til I spoke of my discovery of Simon Willison. He is the author of Datasette a toolset targeted at both individuals and organisations that enables the publication, exploration and analysis of any dataset. Amongst its many features — I particularly like that Datasette lends well to a Jamstack architecture.
Datasette embodies some of the values that I hold regarding data and the internet by enabling individuals to:

The last point is pertinent. Overtime I have acquired at least 2TB of data and I have often wondered about tools/applications that I could build to help explore and analyse this data. Willison introduces this idea of Personal Data Warehouses — for which we could say a meaningful proportion of the global population possesses a de facto data warehouse: we are all producing data which is stored on disparate devices and web platforms. Laws like GDPR attempts to help individuals to own the data held on web platforms… However, this data is not useful unless we can explore and analyse it to gain insight.
Combined with the innovations of LLMs and embeddings, my hope is a next generation of applications will help us in gaining new insights into ourselves, assisting us in the journey of understanding oneself.