However, there are drawbacks to using SQLite in a data production system. Furthermore, SQLite contains most of the SQL commands that are used with bigger engines-so if you know SQLite then you know the basics of every other SQL database All your data is saved onto a single file, making it portable and easy to develop with. If you have ever used a SQL engine in the past, you most likely have been introduced to SQLite, as it is one of the most common database engines. With everything set up, let’s dive into connecting to your local Postgres server! Postgres and the client-server model

If you are running version 2.7 or are on a Windows machine, the commands should still be similar. In our code examples, we will be using Python version 3.6 on a Mac or Linux OS. If not, you can run: pip install psycopg2 Next, you should ensure that you have the psycopg2 library installed. The Postgres wiki has an installation page with guides on the most popular operating systems. The most important thing is to have a local version of Postgres installed on your computer. We like Postgres due to its high stability, ease of accessbility in cloud providers (AWS, Google Cloud, etc), and the fact it is open source! Using the Python library, psycopg2, we will run through an example of how you can create your own table from scratch and then load a data set into a local running Postgres server.īefore we begin, you should make sure you have the necessary tools installed. Specifically, we’ll be focusing on Postgres (or PostgreSQL), one of the biggest open source relational databases.

Relational databases are the most common storage used for web content, large business storage, and, most relevant, for data platforms. In this blog post, we are going to focus on a type of data storage called a relational database. Whether you are a data analyst, data scientist, data engineer, or even a web developer, it is important to know how to store and access your data. The only problem is: no one mentions how you get the data stored in the first place.

You will find hundreds of SQL tutorials online detailing how to write insane SQL analysis queries, how to run complex machine learning algorithms on petabytes of training data, and how to build statistical models on thousands of rows in a database. Data storage is one of (if not) the most integral parts of a data system.

Import data into python