Dave Mason  

I picked an arbitrary database table and exported a million rows to a CSV file and also to a parquet file. It's 234 MB vs 7 MB.

Parquet files are nice!

1+
Chrissy LeMaire  

@DaveMasonDotMe is parquet a binary file? Sorta like how SQLite makes a db file? Seems like it from my quick search!

1
Dave Mason
Follow

@baguette Binary file? Yeah, I think so. I keep seeing that parquet file data is "columnar".

I suspect it is similar in concept to the way Power BI data is stored/compressed via VertiPaq...or the way SQL Server stores/compresses data for columnstore.

· 1· 0· 1
Johan Ludvig Brattås  

@DaveMasonDotMe @baguette Parquet is a binary file that stores the data in columnar mode. So yes, very similar to Columnstore or Vertipaq, but optimized for Spark processing

0
Sign in to participate in the conversation

CounterSocial is the first Social Network Platform to take a zero-tolerance stance to hostile nations, bot accounts and trolls who are weaponizing OUR social media platforms and freedoms to engage in influence operations against us. And we're here to counter it.

Resources

Developers

What is CounterSocial?

Created by potrace 1.15, written by Peter Selinger 2001-2017

counter.social

More…