Apart from the usual advice to "start working on a patch to learn the internals," which is not as easy to do because it's a mature product and most of the low-hanging fruits have been reaped, I'd also suggest checking the extensions shipped with the main source tree under $top/contrib.
Each one is self-contained and usually works with a subset of the internals.
Some good ones: pageinspect, pg_buffercache, pg_prewarm, pg_statstatements, pg_walinspect
The worst part of Postgres (and linux) is the awful mailing list driven development process. I have patches to Postgres that I'd rather just keep porting up every version than to try to figure out how to make Outlook work with a 1980s process.
The internals themselves are quite nicely engineered as you'd expect
Correct. It's corporate email. Imagine the amount of contributions that are gated behind stupid email policies because some fossil mailing list client can't handle modern email in the year of our lord 2025
That can't be true. Now, I haven't used Outlook myself, but I've received enough garbage from that godforsaken relic of a time when MS wanted to EEE email to know that that can't be true.
FWIW, I've posted patches to the PG mailing list just fine with GMail. The web app, that is.
Parent probably means that email clients don't send the attachments (patches) with the appropriate content type (i.e. text/x-patch or text/x-diff) but use application/octet-stream.
> Transaction logs are an essential part of databases because they ensure that no data is lost even when a system failure occurs. They are a history log of all changes and actions in a database system. This ensures that no data is lost due to failures, such as a power failure or a server crash. The log contains sufficient information about each transaction that has already been executed, so the database server can recover the database cluster by replaying the changes and actions in the transaction log in the event of a server crash.
How would you rephrase it? I'm also a non-native English speaker, and the bit you provided sounds very easy to understand because of how it's phrased. There are minor redundancies, but that's all I noticed...
If rrr_oh_man's profile is real (https://bit.ly/48d9o9P), his resume appears inconsistent and unprofessional. The emptier a person is, the more likely they are to criticize others.
I won't comment on the OP using 'horrendous' as an adjective, but that quote seems misleading and unclear to me. Transactions (not transaction logs) are about maintaining [relational] database consistency, not directly about ensuring no data loss. A database engine would rather roll back a partially applied transaction and lose the data in it, than update one table and not update another, breaking the relation between them, and becoming inconsistent/corrupt.
Transaction logs then, are a separate record of these transactions into a log file. The quote says they are 'essential' but three paragraphs later in the link[0] is this: "The [log] mechanism was first implemented in version 7.1"; transaction logs are not essential for RDBMSs and are not essential for ensuring against data loss, they are one part of one larger design for reducing the chance of data loss. (Having a separate log file does not ensure that no data is lost in the event of a crash because the log file could also be lost by the same crash).
"so the database server can recover the database cluster" - the quote has switched from transaction logs as essential, to clustering-with-transaction-logs as essential, without clearly calling that out; there are clustering designs which don't need transaction logs so that isn't essential, and from the PostgreSQL documentation[1]: "It should be noted that log shipping is asynchronous, i.e., the [logs] are shipped after transaction commit. As a result, there is a window for data loss...". Neither transaction logs, nor log shipping clustering, are enough to ensure no data loss. It is possible to improve on that with synchronous_commit[2] (off by default in PostgreSQL) but that can still be designed badly to allow data loss to happen.
The quote is misleading about what transaction logs are for, what situations they can/cannot help with, and what other concerns are involved in guarding against data loss. Given that data loss is a particularly important concern in relational databases compared to some other software and systems, and that there are so many details which need to be considered to reduce risks of data loss, the quote being at all vague or misleading seems worse than it would be if writing about other systems, and worse than it would be if written in the context of 'a high level overview of databases'.
Also recommend this free book: https://postgrespro.com/community/books/internals
Not related at all but the elephant illustration remind me of the Grand Elephant https://www.lesmachines-nantes.fr/en/discover/the-grand-elep...
Apart from the usual advice to "start working on a patch to learn the internals," which is not as easy to do because it's a mature product and most of the low-hanging fruits have been reaped, I'd also suggest checking the extensions shipped with the main source tree under $top/contrib. Each one is self-contained and usually works with a subset of the internals. Some good ones: pageinspect, pg_buffercache, pg_prewarm, pg_statstatements, pg_walinspect
This is an old, but great and regularly updated, source of information about this great RDBMS. A lot of knowledge is accumulated there.
The worst part of Postgres (and linux) is the awful mailing list driven development process. I have patches to Postgres that I'd rather just keep porting up every version than to try to figure out how to make Outlook work with a 1980s process.
The internals themselves are quite nicely engineered as you'd expect
> how to make Outlook work
Well that’s your first problem. Unless you’re using corporate email?
Correct. It's corporate email. Imagine the amount of contributions that are gated behind stupid email policies because some fossil mailing list client can't handle modern email in the year of our lord 2025
> Outlook
> modern email
That can't be true. Now, I haven't used Outlook myself, but I've received enough garbage from that godforsaken relic of a time when MS wanted to EEE email to know that that can't be true.
FWIW, I've posted patches to the PG mailing list just fine with GMail. The web app, that is.
Parent probably means that email clients don't send the attachments (patches) with the appropriate content type (i.e. text/x-patch or text/x-diff) but use application/octet-stream.
You are unable to send a message with an attachment? Or what's the problem specifically?
I don't want to waste my time with a process that has zero benefits to me
Thanks for sharing. I have been digging deeper around database internals and looks amazing.
Check out the CMU history of the DB class, Andy Pavlo is an amazing teacher and you really get a feel for DBs https://www.youtube.com/watch?v=LWS8LEQAUVc&ab_channel=CMUDa... same class from 5 years ago is great because he's like well I can't get in my hotel room so I'll just record from the street https://www.youtube.com/watch?v=SdW5RKUboKc&t=797s&ab_channe.... Tons of good videos to go through on their youtube channel.
> Transaction logs are an essential part of databases because they ensure that no data is lost even when a system failure occurs. They are a history log of all changes and actions in a database system. This ensures that no data is lost due to failures, such as a power failure or a server crash. The log contains sufficient information about each transaction that has already been executed, so the database server can recover the database cluster by replaying the changes and actions in the transaction log in the event of a server crash.
The writing is horrendous
The author is Japanese, cut them some slack.
Or better yet, offer help
It's a little clipped, but very straightforward and easy to follow. Not horrendous.
How would you rephrase it? I'm also a non-native English speaker, and the bit you provided sounds very easy to understand because of how it's phrased. There are minor redundancies, but that's all I noticed...
A bit on the wordy side, but better than the average documentation.
What makes the writing horrendous?
If rrr_oh_man's profile is real (https://bit.ly/48d9o9P), his resume appears inconsistent and unprofessional. The emptier a person is, the more likely they are to criticize others.
I won't comment on the OP using 'horrendous' as an adjective, but that quote seems misleading and unclear to me. Transactions (not transaction logs) are about maintaining [relational] database consistency, not directly about ensuring no data loss. A database engine would rather roll back a partially applied transaction and lose the data in it, than update one table and not update another, breaking the relation between them, and becoming inconsistent/corrupt.
Transaction logs then, are a separate record of these transactions into a log file. The quote says they are 'essential' but three paragraphs later in the link[0] is this: "The [log] mechanism was first implemented in version 7.1"; transaction logs are not essential for RDBMSs and are not essential for ensuring against data loss, they are one part of one larger design for reducing the chance of data loss. (Having a separate log file does not ensure that no data is lost in the event of a crash because the log file could also be lost by the same crash).
"so the database server can recover the database cluster" - the quote has switched from transaction logs as essential, to clustering-with-transaction-logs as essential, without clearly calling that out; there are clustering designs which don't need transaction logs so that isn't essential, and from the PostgreSQL documentation[1]: "It should be noted that log shipping is asynchronous, i.e., the [logs] are shipped after transaction commit. As a result, there is a window for data loss...". Neither transaction logs, nor log shipping clustering, are enough to ensure no data loss. It is possible to improve on that with synchronous_commit[2] (off by default in PostgreSQL) but that can still be designed badly to allow data loss to happen.
The quote is misleading about what transaction logs are for, what situations they can/cannot help with, and what other concerns are involved in guarding against data loss. Given that data loss is a particularly important concern in relational databases compared to some other software and systems, and that there are so many details which need to be considered to reduce risks of data loss, the quote being at all vague or misleading seems worse than it would be if writing about other systems, and worse than it would be if written in the context of 'a high level overview of databases'.
[0] http://www.interdb.jp/pg/pgsql09.html
[1] https://www.postgresql.org/docs/17/warm-standby.html
[2] https://www.postgresql.org/docs/17/warm-standby.html#SYNCHRO...