--- title: PSQL (PostgreSQL Client) homepage: https://www.postgresql.org/ tagline: | psql: REPL for Postgres that supports standard TLS (+SNI,ALPN), .pgpass, and .psqlrc --- To update or switch versions, run `webi psql@stable` (or `@v17.0`, `@beta`, etc). ### Files These are the files / directories that are created and/or modified with this install: ```text ~/.config/envman/PATH.env ~/.local/opt/psql/ ~/.pgpass ~/.psqlrc ~/.config/psql/psqlrc.sql ``` ## Cheat Sheet > `psql` serves as a model for what a great PostgreSQL Client should be: > > - gives you a nice REPL to run queries > - manages authentication > - allows table or list view > - preloads user-specific settings For localhost / private networking: ```sh psql "postgres://postgres:postgres@localhost:5432/postgres" psql "postgres://db-xxxx:secret123@pg-1.example.com:5432/db-xxxx" ``` For remote / public networks: \ (with `sslmode` & `sslnegotiation` for standard TLS and SNI, with ALPN set to 'postgresql') ```sh psql "postgres://db-xxxx@pg-1.example.com:5432/db-xxxx?sslmode=require&sslnegotiation=direct" ``` ## Table of Contents - Server vs Client & PG Essentials - Vertical Rows - .pgpass - .psqlrc - How to Import / Export CSV - How to Backup & Restore - How to Create a Table - Session Variables (& Encryption Keys) ### Where to Find the Postgres Server Cheat Sheet This is exclusively a client-side cheat sheet. For the server-side / administrative cheat sheet, see [The Postgres (Server) Cheat Sheet](../postgres/). For a collection of other helpful scripts, see [PG Essentials](https://github.com/bnnanet/pg-essentials): - psql-example-connect - psql-backup - psql-store-credential ### How to View Rows Vertically - use `\gx` instead of `;` for a single query - use `\x` to toggle for all queries ```sql SELECT id, character_name, description FROM character_descriptions \gx ``` ```text -[ RECORD 1 ]--+------------------------------------------------------------------------------------------------------------------ id | e17709d3-6e5a-4baa-91cc-8d5267815d5e character_name | Harry Potter description | A teenager with a chronic case of "Chosen One" syndrome, who manages to stay alive by sheer luck. -[ RECORD 2 ]--+------------------------------------------------------------------------------------------------------------------ id | 68646acf-9468-4006-83a7-8ee748df4ca5 character_name | Spider-Man description | The neighborhood's most responsible guy, who balances high school stress with the casual task of saving the city. ``` - similar to `\G` in MySQL. ### How to use `~/.pgpass` for Passwords Postgres passwords are stored in `~/.pgpass`. You can use [psql-store-credential](https://github.com/bnnanet/pg-essentials) to manage `~/.pgpass`, or manage it manually: ```sh touch ~/.pgpass chmod 0600 ~/.pgpass ``` ```sh # export PGPASSFILE='/Users/aj/.pgpass' # hostname:port:database:username:password localhost:5432:postgres:postgres:postgres pg-1.example.com:5432:db-xxxx:secret123 localhost:*:*:postgres:postgres *:*:db-xxxx:db-xxxx:secret123 *:*:*:postgres ``` - this _ONLY_ supplies a password - not a default username or db name - the _Database Name_ and _User Name_ are typically the same \ (as per `samename` in `hba.conf`) - all but the password can `*` wildcards - the first line to match (_NOT_ the most specific) will used for password ### How to setup `~/.psqlrc` for Per-DB history Allows you to keep per-database history and settings, such as encryption keys. ```sh mkdir -p ~/.config/psql/ touch ~/.psqlrc touch ~/.config/psql/psqlrc.sql chmod 0600 ~/.psqlrc chmod 0700 ~/.config ~/.config/psql/ chmod 0600 ~/.config/psql/psqlrc.sql ``` `~/.psqlrc` ```sql \i ~/.config/psql/psqlrc.sql ``` `~/.config/psql/psqlrc.sql`: ```sql -- psql meta-commands: https://www.postgresql.org/docs/current/app-psql.html -- -- Per-DB Configuration -- \set HISTFILE ~/.config/psql/ :DBNAME /history \set confdir ~/.config/psql/ :DBNAME \set dbrc :confdir /psqlrc.sql \if `mkdir -p :confdir && chmod 0700 :confdir && echo n || echo y` \echo [WARN] could not create :confdir \set HISTFILE ~/.config/psql/history \else \if `test -f :dbrc || touch :dbrc && chmod 0600 :dbrc && echo n || echo y` \echo [WARN] could not create :dbrc \else \echo loading :dbrc \i :dbrc \endif \if `test -f :HISTFILE || touch :HISTFILE && chmod 0600 :HISTFILE && echo n || echo y` \echo [WARN] could not create :HISTFILE \set HISTFILE ~/.config/psql/history \endif \endif \unset :dbrc -- -- Session Preferences -- -- ignore space-prefixed commands and duplicates \echo using :HISTFILE for command history \set QUIET on \set HISTCONTROL ignoreboth \set ON_ERROR_ROLLBACK interactive \set COMP_KEYWORD_CASE upper \pset pager off \pset null '(null)' -- set to YOUR timezone SET TIME ZONE 'America/Denver'; \unset QUIET \echo '' ``` - `HISTCONTROL ignoreboth` causes lines starting with a space and duplicate lines to be omitted from history ### How to Work with CSVs - `\copy` saves client-side (locally, relative to `psql`) - MUST be on a SINGLE LINE (no newlines) - (use a temporary view for for queries that don't easily fit on a line) - `COPY` saves server-side (remote, relative to `postgres`) #### How to Export to CSV ```sql \copy "character_descriptions" TO './character_descriptions.csv' WITH CSV HEADER; ``` ```sql CREATE TEMP VIEW "character_descriptions_csv" AS SELECT "character_name", "description" FROM "character_descriptions"; \copy (SELECT * FROM "character_descriptions_csv") TO './character_descriptions.csv' WITH CSV HEADER; ``` #### How to Import from CSV ```sql \copy "character_descriptions"("id", "character_name", "description") FROM './character_descriptions.csv' WITH (FORMAT csv, HEADER); ``` ### How to Backup & Restore To backup in a way that will be easy to restore: - save the schema separately from the data - don't include database-specific roles or permissions - store the password in `~/.pgpass` as described above You can use a helper script like [psql-backup](https://github.com/bnnanet/pg-essentials), or create your own: Given these credentials: ```sh my_user="db_xxxx" my_db="db_xxxx" my_host="pg-1.example.com" my_port="5432" ``` And these [`pg_dump`](https://www.postgresql.org/docs/current/app-pgdump.html) commands: ```sh pg_dump --no-privileges --no-owner --schema-only --clean \ --username "$my_user" --no-password --host "$my_host" --port "$my_port" \ -f ./"$my_db".schema.drop.sql "$my_db" >&2 pg_dump --no-privileges --no-owner --schema-only \ --username "$my_user" --no-password --host "$my_host" --port "$my_port" \ -f ./"$my_db".schema.sql "$my_db" >&2 pg_dump --no-privileges --no-owner --data-only \ --username "$my_user" --no-password --host "$my_host" --port "$my_port" \ -f ./"$my_db".data.sql "$my_db" ``` You'll get your data is this format: ```text db_xxxx.schema.drop.sql # will replace (DELETE) all tables with empty tables db_xxxx.schema.sql # will create new empty tables db_xxxx.data.sql # will load data ``` To restore / copy to another database: ```sh new_user="db_yyyy" new_db="db_yyyy" psql "postgres://$new_user@$my_host:$my_port/$new_db" < ./db_xxxx.schema.sql psql "postgres://$new_user@$my_host:$my_port/$new_db" < ./db_xxxx.data.sql ``` Or use [`pg_restore`](https://www.postgresql.org/docs/current/app-pgrestore.html). See the examples at: - https://github.com/bnnanet/pg-essentials?tab=readme-ov-file#psql-backup - https://github.com/therootcompany/pg-xzbackup.sh ### How to Create a Table ```sql CREATE TABLE "example_table" ( -- Primary Key using gen_random_uuid() "id" UUID PRIMARY KEY DEFAULT gen_random_uuid(), -- Numeric types "small_int_column" SMALLINT, "positive_integer_column" INTEGER CHECK ("positive_integer_column" > 0), "big_int_column" BIGINT, "decimal_column" DECIMAL(10, 2), "numeric_column" NUMERIC(15, 3), "real_column" REAL, "double_precision_column" DOUBLE PRECISION, "serial_column" SERIAL NOT NULL UNIQUE, "bigserial_column" BIGSERIAL NOT NULL UNIQUE, -- Monetary type "money_column" MONEY, -- Character types "char_column" CHAR(10), "varchar_column" VARCHAR(255), "text_column" TEXT, -- Binary types "bytea_column" BYTEA, -- Date/time types "timestamp_column" TIMESTAMP, "timestamp_tz_column" TIMESTAMP WITH TIME ZONE, "date_column" DATE, "time_column" TIME, "time_tz_column" TIME WITH TIME ZONE, "interval_column" INTERVAL, -- Boolean type "boolean_column" BOOLEAN, -- UUID type "uuid_column" UUID, -- JSON types "json_column" JSON DEFAULT '{}', "jsonb_column" JSONB, -- Array types "int_array_column" INTEGER[], "text_array_column" TEXT[], -- Geometric types "point_column" POINT, "line_column" LINE, "lseg_column" LSEG, "box_column" BOX, "path_column" PATH, "polygon_column" POLYGON, "circle_column" CIRCLE, -- Network address types "cidr_column" CIDR, "inet_column" INET, "macaddr_column" MACADDR, "macaddr8_column" MACADDR8, -- Bit string types "bit_column" BIT(8), "varbit_column" BIT VARYING(8), -- Text search types "tsvector_column" TSVECTOR, "tsquery_column" TSQUERY, -- XML type "xml_column" XML, -- Range types "int4range_column" INT4RANGE, "numrange_column" NUMRANGE, "tsrange_column" TSRANGE, "tstzrange_column" TSTZRANGE, "daterange_column" DATERANGE, -- Hstore type "hstore_column" HSTORE, -- Unique constraint (example) CONSTRAINT "unique_example_table_columns" UNIQUE ("positive_integer_column", "varchar_column") ); COMMENT ON COLUMN "example_table"."id" IS 'Unique identifier for the example table'; ``` ### How to use Session Variables Given the example `psqlrc` above which creates per-db history and config files, you can create a config file with the session variables you'd like to use in queries: `~/.config/psql/db_xxxx/psqlrc.sql`: ``` -- ex: add conventional "my" extension with client params for pgp and raw encryption SET SESSION "my"."client_id" = '12345678'; -- note: MUST be cast to ::bytea explicitly -- SELECT current_setting('my.aes_128_key')::bytea; SET SESSION "my"."aes_128_key" = E'\\xdeadbeefbadc0ffee0ddf00dcafebabe'; SET SESSION "my"."pgp_password" = 'zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo right'; ``` ```sql SELECT current_setting('my.aes_128_key')::bytea; SELECT current_setting('my.pgp_password'); ``` - Bound to the _CURRENT LOGIN SESSION_ only - Can be used as a _VALUE_, anywhere - such as in _FUNCTIONS_ and _VIEWS_ - Must must be scoped "extension"."var", where "extension" is arbitrary \ (conventionally "my", but can be anything that isn't already a schema) - Can also be used programmatically by clients in an `exec()` or `query()` #### Example Raw AES Encryption This shows a full encrypt and decrypt example, using `current_setting()` to get a key. ```sql -- raw encrypt / decrypt example WITH "aes_key_cte" AS ( SELECT current_setting('my.aes_128_key')::bytea AS "aes_key", 'sensitive data (raw)' AS "plain_original" ), "raw_example_table_cte" AS ( SELECT "aes_key", "plain_original", encrypt(convert_to("plain_original", 'UTF8'), "aes_key", 'aes') AS "raw_enc_column" FROM "aes_key_cte" ) SELECT "plain_original", "raw_enc_column", convert_from(decrypt("raw_enc_column", "aes_key", 'aes'), 'UTF8') AS "plain_decrypted", convert_from(decrypt_iv( "raw_enc_column", "aes_key", E'\\x00000000000000000000000000000000', 'aes'), 'UTF8') AS "plain_decrypted_iv" FROM "raw_example_table_cte" ; ``` In practice, using a simple and efficient SQL functions can help abstract away the tedious bits. See: - https://www.postgresql.org/docs/current/pgcrypto.html#PGCRYPTO-RAW-ENC-FUNCS #### Example PGP AES Encryption ```sql -- pgp_sym encrypt / decrypt example WITH "pgp_pass_cte" AS ( SELECT 'sensitive data (pgp)' AS "plain_original", current_setting('my.pgp_password') AS "pgp_pass" ), "pgp_example_table_cte" AS ( SELECT -- pgp_sym_encrypt(data text, psw text [, options text ]) returns bytea -- pgp_sym_encrypt_bytea(data bytea, psw text [, options text ]) returns bytea pgp_sym_encrypt( 'sensitive data (pgp)', "pgp_pass", 'cipher-algo=aes128, unicode-mode=1' ) AS "pgp_enc_column" FROM "pgp_pass_cte" ) SELECT "plain_original", "pgp_enc_column", -- pgp_sym_decrypt(msg bytea, psw text [, options text ]) returns text -- pgp_sym_decrypt_bytea(msg bytea, psw text [, options text ]) returns bytea pgp_sym_decrypt( "pgp_enc_column", "pgp_pass", 'cipher-algo=aes128, unicode-mode=1' ) AS "plain_decrypted" FROM "pgp_example_table_cte" CROSS JOIN "pgp_pass_cte" ``` See: - https://www.postgresql.org/docs/current/pgcrypto.html#PGCRYPTO-PGP-ENC-FUNCS