Managed databases with sqlc vet
Getting Postgres logs in a GitHub Action https://brandur.org/fragments/postgres-logs-in-github-actions | https://lobste.rs/s/rlkybm #databases #testing
I'm planing on setting up #postgresql in a @docker container. In the past, I've stayed away from using #databases in a container.It seemed like #containers tech was too immature to rely on for a database. But it seems like perhaps it has matured at least for a development instance of a #db. I'm still going to use a hosted solution for a production database. I'm using #GCP for this project) Wonder if folks have any advice on how to do it right. @planetpostgresql
You may have read my post about big data not really being a thing:
but you may have remained skeptical. In that case, here's someone else saying the exact same thing:
Marketing angle aside, the conclusions are the same. Big data is not real and even if it was, it's not useful.
Any #postgresql server admins out there? Successfully running a decently large cluster for quite a while but have never quite figured out if there is server side tuning that can be brought to bear to weed out LWLock:buffer_mapping. Sure, smaller batches, or table sharding, but that's software side. Any server settings tweaks? #databases #dbadmins
How do databases execute expressions?
64-Bit Bank Balances ‘Ought to be Enough for Anybody’?
What if OpenDocument used SQLite? (2014)
We Need A Standard Layered Image Format (2013)
Using Janet as Database
PostgreSQL 16 Released https://www.postgresql.org/about/news/postgresql-16-released-2715/ | https://lobste.rs/s/py94bo #databases
#postgresql 16 has been released 🎉
Very glad to see a successful PGDay UK take place. The UK #PostgreSQL #community is thriving and there is huge potential to connect, learn from & work with each other. Let's keep growing, and potentially take this to other places in the UK.
Congratulations to all the organisers, volunteers and speakers for an excellent #conference, and a big thank you to Alice Keane and Mostafa Zakaria for their great presence at the EDB booth.
Datalite: A Simple Datalog Built Around SQLite in Python
I just really like writing #databases in docs… i dunno why but organising notes in a table and adding columns for various metadata really speaks to me.
I had a page for job listings and house hunting, I’m not cases Notion flexible enough to support changing needs in terms of the fields I was storing and the way I thought about the data.
Check out the Leanpub Frontmatter podcast interview with Andy Grove, Author of How Query Engines Work: An Introductory Guide https://leanpub.com/podcasts/frontmatter/andy-grove-17-02-21 #books #databases #podcasts
#Databases in the #Microservices world: "Why do we refactor apps into microservices then use the same #monolith #datastore our grandparents used?" Join us #online for #TwinCities #dotNET User Group at https://www.meetup.com/tcdnug/events/295749203/ and grab the slides at https://robrich.org/slides/databases-in-the-microservices-world/#/
The authors argue that despite increasing specialization, databases share the same components. By relying on standards, you can develop a composable system with:
- your preferred language interface (#SQL, #Ibis, #dplyr, ...). The query is converted into an intermediate representation (like #substrait)
- a query optimizer
- an execution engine
- an execution runtime
The Top Programming Languages 2023
Well, I think it could be nice to talk a bit about myself, so here it goes (it is a long one, be warned)
My (still ongoing) journey from engineering to programming: Prologue
As can be seen in my profile bio, my name is Iago Andrade, and I am a Brazilian. What can't be seen there, though, is that I'm a mechanical engineer, and I have worked as a mechanical designer since 2018. This year (2023) I decided to pursue an old desire, which was to be able to work with #programming.
From the very first year at university, I fell in love with programming, when we were introduced to #C. Most of my friends were not really interested in that, and thought it was too difficult, but I was fascinated by the possibilities of what could be done with it.
At first, we used it mostly to solve dificult mathematical problems through iterative process, but my curiosity led me to learn more than just what was taught in classes. We were also later introduced to MATLAB and EES, and I even learned a bit of #Python in order to help a friend of mine.
But as the course became more and more time consuming, I started to drift away from that passion, and focus more on the other disciplines, and only sparingly did I delve into anything related to programming.
As I graduated, I soon got a #job in mechanical design, modeling industrial equipment and parts and its drafts for production. As I had little time for studying, programming was put aside for a while, and all the focus was put into this job.
I learned a lot at this job, and managed to build a lot of resources for accelerating, and even automating tasks, and that kept me reminded of how much I liked to solve problems like these. I even got to the point of learning some #bash script in order to automate the creation of folders in the company's server (which did not end up very well, due to the lack of some quotes hehe). Though the script caused some problems, it was solved in the end (and I learned the important lesson of debugging everything very thoroughly).
Reigniting an old passion
At some point between then and now, I learned of the existence of a "new" game engine called #Godot. I had never made any #games, and had never used a game engine before either, but I had had some contact with #FOSS (free and open source softwares), like #blender and #inkscape, and that one caught my attention, since I had always wanted to create games. Since it was free, super lightweight, I decided to give it a go. It felt so good to learn that tool, and programming in it reminded me of how much I liked to code.
Back then, a friend of mine was also interested in making a game, and he wanted to be more focused on the story part of it, which was perfect since I wanted to focus on the programming and possibly the art and music for it. We started to work on the idea, got a ton of references, and started building the overall scope and story, until one day we lost momentum and just couldn't get back to it. Our jobs were sucking too much energy, and we decided to put it aside until we had more time to do it properly.
#SQL and the leap of faith
After some more years of working with 3D modeling, the company where I was working decided to implement an ERP system. The problem was they didn't have anyone from the inside who was qualified for some of the tasks that would appear, and they tasked me with dealing with those problems.
I had never worked with an ERP system, but had some idea of how #databases worked, and that helped a lot in making some decisions early on (like not entrying all data manually). I had to learn how to bulk clean old data from another system they had used before, which had been fed by many different people, with a lot of mistakes and inconsistencies.
After that, I had to import all that data into the system, and I decided it was time to learn SQL so I could query the database properly when I needed information. And that once again triggered my interest in programming, leading to my decision to make new templates for the engineering team.
These templates used programming for automating some of the tasks, in a language specific for the modeling software,reducing many of the steps in the project pipeline, and making it easier to input new equipment in the system.
As I became more and more #tired of some problems at work, and got more interested in programming again, I made the decision of focusing in relearning the languages I had contact with, and while searching for resources, I was informed that #Harvard offered this course online, called #CS50
For a few months I tried, and was unable to #study in my spare time. So ultimately, I decided to #quit my job, finish CS50, and start a career in programming.
Paying the #bills
With only the final project from the course left, money was running short, and bills were still appearing, so I had to start looking for jobs, and I ended up getting another job in mechanical design. The difference now is that it is much closer to home, so I will be able to finish my final project on my spare time, and keep looking for #junior roles in programming.
Epilogue: The Final Project
Since this toot is ridiculously long already, I may as well talk about the final project for whoever got to this part haha
Explaining The Postgres Meme https://www.avestura.dev/blog/explaining-the-postgres-meme | https://lobste.rs/s/1rbawx #databases #math
Data scientists and cancer specialists demonstrate a data encryption technique that makes possible real-world cancer data sharing while preserving individual privacy.
Scaling Postgres to 1 Million Connections
From the publisher: “…how these #women developed communication #networks, #databases & digital #archives… #activists brought people and their visions of #justice together… media technologies animate the collective and unspectacular #labor that sustains social movements, including their #antiracist and #trans-inclusive endeavors.”
So I'm back from sailing and today is PGSQL Phriday (weird spelling I know) and the topic for today's community Postgres blogging event is "Partitioning vs. Sharding", a topic that is right up my alley!
So here is my blog contribution on Understanding Partitioning and Sharding in Postgres, featuring the Citus database extension in the sharding section of course!
A bit of a programmer rant...
People wonder why I like ORMs even when they're unnecessary. Firstly, I've never liked SQL. I think that writing queries to a RDBMS is something that a computer should do, akin to compilation. In the few times when extreme optimization is warranted, low level code can be generated to suit that specific case. In other times, ORMs usually provide a more natural interface to data that increases readability and code flow.
Research in programming Wikidata https://en.m.wikiversity.org/wiki/Research_in_programming_Wikidata | https://lobste.rs/s/bsn3gy #book #databases #science
What is database-as-a-service (DBaaS)? https://silverliningsinfo.com/apps-services/what-database-service-dbaas by @sjvn
#Cloud Databases as a Service make using #databases much easier, but that's not the same thing as easy.
Yet another question... what's the Squarespace/WordPress of databases?
Been bumping into all these cool new museums/independent research orgs people are starting up and I really feel they deserve better than the bloated crap sold at museum expos that the big institutions are currently saddled with...
I got nerdsniped by @tef & @sushee to demonstrate how relatively easy it is to build a CRAQ (https://timilearning.com/posts/mit-6.824/lecture-9-craq/) system atop @couchdb — Showing that as much as CouchDB is a distributed database in itself, it is also a very nice toolkit to build other kinds of distributed databases with it :)
The demo is in Node.js with some private JS/HTTP API glue that is not relevant to the demo: https://gist.github.com/janl/d0ba3544631e054e59fbd5f4d3ee96f8
Let me know if you ahave any questions.
The topic for this month's podcast for developers who love Postgres (aka "Path To Citus Con") is:
> My favorite ways to learn more about Postgres
> Live podcast recording will happen on Discord on Wed 12 July @ 10am PDT
Guests = @ryanbooz & Grant Fritchey
Hosts = myself & Pino de Candia
Join us! And please boost so more people discover it! Calendar invite w/details on how to join https://aka.ms/PathToCitusCon-Ep05-cal
Doing an online course on #Databases - I'm a bit rusty on #SQL queries. I'll be trying #Docker for the first time to create a DB test environment. Weird that I haven't tried it before, suppose having access to your own VM means less faffing around via a desktop machine. Also probably put off by all the girning I see on social media relating to Docker. Almost every day someone in my feed will complain about some setting or other that caused calamity and lost time...what delights/horrors await? 🤯
Got some time this Saturday? Check out Cory Doctorow's linkblog @pluralistic long thread: https://mamot.fr/@pluralistic/110520863461422021 also https://pluralistic.net/2023/06/10/in-the-dumps/#do-the-humpty-dump
Primate Genome A.I. Model IDs Human Disease Variants
Bruce Momjian of EDB is a prolific PostgreSQL presenter 🔥 and the organizers of Citus Con: An Event for Postgres were thrilled he gave a talk at Citus Con.
So today's Citus Con video of the day is from Bruce! If you are looking for a solid primer on Postgres & the Artificial Intelligence landscape, Bruce's #CitusCon talk 📺🎙️ is the place to start.
If you want to learn about the key to scalable (horizontally scalable) & fault-tolerant PostgreSQL, watch this talk (and demo) about Citus and Patroni.
Writing a chat application in Django 4.2 using async StreamingHttpResponse, Server-Sent Events and…
It's all about Parallelism in PostgreSQL. With a lovely sock analogy to boot. Thomas is such a good explainer & teacher, hope y'all enjoy it.
Collaboration to Boost Rare Disease Clinical Trials
I do rather love being able to run tail¹ on my database tables² as I work on building Domain³ with Kitten⁴ ;)
The “Build Your Own Database” book is finished https://build-your-own.org/blog/20230420_byodb_done/ | https://lobste.rs/s/gibx9w #databases #go #programming
Ouch, the startup I joined last year could not get funding, so I need to get a new job... I would love to continue working with #rustlang but I'm also experienced in software architecture, #java, relational and graph #databases... I'm working remotely from the South of France, I've worked in small startups and big companies so I can adapt. Just DM me if you need somebody like me!
My product team at Google just released a downloadable edition of #AlloyDB, its #PostgreSQL-compatible database engine loaded with lots of neat features like an index advisor and a columnar engine and so on and suchlike.
It's in preview! You can install it on any newish RHEL or Debian-like and try it out for free.
I co-authored the docs, by gar.
ICYMI, I'm speaking about debugging in production at #QConLondon later this month.
The lovely organisers have set me up with a discount code, so if you'd like an extra £50 off your ticket, use code ChrisSQUK2350 when buying your ticket (https://qconlondon.com/registration/event/1)! Early bird ends on the 6th.
Really excited about this one. Hope to see you there ✌🏻💖