KEMBAR78
What’s New In PostgreSQL 9.3 | PDF
What’s New In PostgreSQL 9.3 ?
Pavan Deolasee
Nov 16, 2013
India PostgreSQL UserGroup MeetUp
Who am I ?


Pavan Deolasee
–



http://www.linkedin.com/in/pavandeolasee

Contributor to PostgreSQL and Postgres-XC Global
DevelopmentBest known for development of Heap-OnlyTuple (HOT) feature of PostgreSQL 8.3 release
–

Contributed several other enhancements to PostgreSQL
and derivatives

–

Contributed to Postgres-XC's architectural design and
implementation of several features



Currently works at NTT Data as a consultant



Previously worked for EnterpriseDB and Symantec/Veritas
Credits
●

Magnus Hagander

●

Michael Paquier

●

Bruce Momjian

●

Of course, all the developers
PostgreSQL Recap
PostgreSQL
–

is world's most advanced open source database

–

is very stable

–

is fully compliant with ANSI SQL

–

supports foreign key, check constraints

–

supports various kinds of indexes

–

supports inheritance

–

is fully extensible (data types, procedural languages etc)

–

will recover your database in case of server failure
PostgreSQL Recap
PostgreSQL
–

uses write-ahead-logs for durability and recover your database
in case of server failure

–

built-in log based streaming synchronous/asynchronous
replication

–

file system level backups and archive recovery

–

point-in-time recovery

–

hot standby

–

upgrade in place

–

full-text search
What’s New Really in 9.3 ?
●

SQL features

●

Performance features

●

DBA features

●

Many more general features
Auto Updatable Views
●

Simple views are now updatable i.e. you can run
INSERT/UPDATE/DELETE queries against a view that uses a
single base relation

●

Many restrictions apply

●

More complex views can be updated via RULEs mechanism
Materialized Views
●

●

Query is executed once and result is materialized on a stable
storage
Views must be refreshed manually

CREATE MATERIALIZED VIEW
..
AS query [ WITH [ NO ] DATA ]
LATERAL Support
●

●

Subqueries appearing in FROM can be preceded by the key
word LATERAL.
This allows them to reference columns provided by
preceding FROM items.
–

●

Without LATERAL, each subquery is evaluated
independently and so cannot cross-reference any
other FROM item.

LATERAL is primarily useful when the cross-referenced
column is necessary for computing the row(s) to be joined.
–

A common application is providing an argument value for
a set-returning function
LATERAL Example
SELECT m.name FROM
manufacturers m LEFT JOIN
LATERAL get_product_names(m.id) pname
ON true WHERE pname IS NULL;
Range Type Enhancements
●

●

SP-Gist index support for range types
Statistics collection for range types thus improving query
planning and execution
JSON Enhancements
●

●

●

●

Operators and functions to extract elements from JSON
values
Allow JSON values to be converted into records
Functions to convert scalars, records and hstore values to
JSON
Aggregates
Foreign Data Wrappers
●

●

Several new features
PostgreSQL FDW (postgres_fdw) is available in core now
(forget the old dblink)

●

Writable FDWs are now reality

●

Many FDWs are already available
–

Hbase

–

MongoDB

–

MySQL

–

Oracle
COPY FREEZE
●

●

Load tuples in frozen state to avoid another rewrite of the
table
A few restrictions apply
–

●

A table must be truncated or created in the same
transaction loading the frozen data

A very useful tip to avoid significant IO overhead while
working with large tables
Background Workers
●

●

●

Step towards parallel query execution which is still a release
or two away
Background workers can be used with 9.3 if you have enough
guts to write some cool C code
Look at contrib/worker_spi for some sample code
Posix/mmap Shared Memory
●

●

A step away from SysV Shared memory
Most unix-like systems support Posix or mmap and
PostgreSQL will use them by default
–

●

A small SysV shared memory segment is still allocated for
book keeping purposes

No need to tune your kernel parameters or hit runtime issues
because of hitting kernel limits
Page Level Checksums
●

●

A great feature for checking data consistency
Can be turned on (default is of) at the initdb time and must
stay that way
–

●

Comes at a cost

Errors can’t be corrected, but corruptions can be detected
very early in the cycle
Foreign Key Locking
●

●

Improved concurrency since non-key updates don’t block
foreign key checks
New lock levels
–
–

●

●

FOR KEY SHARE
FOR NO KEY UPDATE

Foreign key triggers now take FOR KEY SHARE locks
DMLs updating non key columns should take FOR NO KEY
UPDATE locks thus avoiding blocking
Event Triggers
●

They are more like DDL triggers
–

●

●

●

Triggers are invoked when new database objects are
created, updated or dropped

Will be useful for logging, monitoring and even some
replication solutions
Three types of triggers - ddl_command_start,
ddl_command_end, sql_drop
More support expected in later releases
Parallel pg_dump
●

Faster pg_dump by using multiple processes
–
–

●

Better utilization of multiple cores
Multiple tables are dumped in parallel

Remember pg_restore can already run parallel jobs
Streaming-only Remastering
●

"Remastering" is the process whereby a replica in a set of
replicas becomes the new master for all of the other replicas.
–

Master M1 is replicating to replicas R1, R2 and R3.

–

Master M1 needs to be taken down for a hardware
upgrade.

–

The DBA promotes R1 to be the master.

–

R2 and R3 are reconfigured & restarted, and now replicate
from R1
Streaming-only Remastering
●

●

In prior versions, it wasn’t possible to stream from new
master without taking a fresh backup
That restriction is now gone
Arch Independent Streaming
●

Stream WAL from diferent OS and diferent architecture
–

pg_basebackup and pg_receivexlog can now work cross
platform

–

You can’t apply WAL from diferent architecture, but this is
still very useful for centralized backup etc
Many More
●

pg_basebackup conf setup (-R option)

●

Logging enhancements

●

Recursive views

●

COPY PIPE

●

Include_dir directive

●

Array enhancements
Resources
●

Release Notes
–

●

Planet PostgreSQL
–

●

http://planet.postgresql.org/

Documentation
–

●

http://www.postgresql.org/docs/9.3/static/release-9-3.html

http://www.postgresql.org/docs/9.3/static/index.html

Source Code
–

http://git.postgresql.org/gitweb/
Thank you
Pavan Deolasee
pavan.deolasee@gmail.com
http://www.linkedin.com/in/pavandeolasee

What’s New In PostgreSQL 9.3

  • 1.
    What’s New InPostgreSQL 9.3 ? Pavan Deolasee Nov 16, 2013 India PostgreSQL UserGroup MeetUp
  • 2.
    Who am I?  Pavan Deolasee –  http://www.linkedin.com/in/pavandeolasee Contributor to PostgreSQL and Postgres-XC Global DevelopmentBest known for development of Heap-OnlyTuple (HOT) feature of PostgreSQL 8.3 release – Contributed several other enhancements to PostgreSQL and derivatives – Contributed to Postgres-XC's architectural design and implementation of several features  Currently works at NTT Data as a consultant  Previously worked for EnterpriseDB and Symantec/Veritas
  • 3.
    Credits ● Magnus Hagander ● Michael Paquier ● BruceMomjian ● Of course, all the developers
  • 4.
    PostgreSQL Recap PostgreSQL – is world'smost advanced open source database – is very stable – is fully compliant with ANSI SQL – supports foreign key, check constraints – supports various kinds of indexes – supports inheritance – is fully extensible (data types, procedural languages etc) – will recover your database in case of server failure
  • 5.
    PostgreSQL Recap PostgreSQL – uses write-ahead-logsfor durability and recover your database in case of server failure – built-in log based streaming synchronous/asynchronous replication – file system level backups and archive recovery – point-in-time recovery – hot standby – upgrade in place – full-text search
  • 6.
    What’s New Reallyin 9.3 ? ● SQL features ● Performance features ● DBA features ● Many more general features
  • 7.
    Auto Updatable Views ● Simpleviews are now updatable i.e. you can run INSERT/UPDATE/DELETE queries against a view that uses a single base relation ● Many restrictions apply ● More complex views can be updated via RULEs mechanism
  • 8.
    Materialized Views ● ● Query isexecuted once and result is materialized on a stable storage Views must be refreshed manually CREATE MATERIALIZED VIEW .. AS query [ WITH [ NO ] DATA ]
  • 9.
    LATERAL Support ● ● Subqueries appearingin FROM can be preceded by the key word LATERAL. This allows them to reference columns provided by preceding FROM items. – ● Without LATERAL, each subquery is evaluated independently and so cannot cross-reference any other FROM item. LATERAL is primarily useful when the cross-referenced column is necessary for computing the row(s) to be joined. – A common application is providing an argument value for a set-returning function
  • 10.
    LATERAL Example SELECT m.nameFROM manufacturers m LEFT JOIN LATERAL get_product_names(m.id) pname ON true WHERE pname IS NULL;
  • 11.
    Range Type Enhancements ● ● SP-Gistindex support for range types Statistics collection for range types thus improving query planning and execution
  • 12.
    JSON Enhancements ● ● ● ● Operators andfunctions to extract elements from JSON values Allow JSON values to be converted into records Functions to convert scalars, records and hstore values to JSON Aggregates
  • 13.
    Foreign Data Wrappers ● ● Severalnew features PostgreSQL FDW (postgres_fdw) is available in core now (forget the old dblink) ● Writable FDWs are now reality ● Many FDWs are already available – Hbase – MongoDB – MySQL – Oracle
  • 14.
    COPY FREEZE ● ● Load tuplesin frozen state to avoid another rewrite of the table A few restrictions apply – ● A table must be truncated or created in the same transaction loading the frozen data A very useful tip to avoid significant IO overhead while working with large tables
  • 15.
    Background Workers ● ● ● Step towardsparallel query execution which is still a release or two away Background workers can be used with 9.3 if you have enough guts to write some cool C code Look at contrib/worker_spi for some sample code
  • 16.
    Posix/mmap Shared Memory ● ● Astep away from SysV Shared memory Most unix-like systems support Posix or mmap and PostgreSQL will use them by default – ● A small SysV shared memory segment is still allocated for book keeping purposes No need to tune your kernel parameters or hit runtime issues because of hitting kernel limits
  • 17.
    Page Level Checksums ● ● Agreat feature for checking data consistency Can be turned on (default is of) at the initdb time and must stay that way – ● Comes at a cost Errors can’t be corrected, but corruptions can be detected very early in the cycle
  • 18.
    Foreign Key Locking ● ● Improvedconcurrency since non-key updates don’t block foreign key checks New lock levels – – ● ● FOR KEY SHARE FOR NO KEY UPDATE Foreign key triggers now take FOR KEY SHARE locks DMLs updating non key columns should take FOR NO KEY UPDATE locks thus avoiding blocking
  • 19.
    Event Triggers ● They aremore like DDL triggers – ● ● ● Triggers are invoked when new database objects are created, updated or dropped Will be useful for logging, monitoring and even some replication solutions Three types of triggers - ddl_command_start, ddl_command_end, sql_drop More support expected in later releases
  • 20.
    Parallel pg_dump ● Faster pg_dumpby using multiple processes – – ● Better utilization of multiple cores Multiple tables are dumped in parallel Remember pg_restore can already run parallel jobs
  • 21.
    Streaming-only Remastering ● "Remastering" isthe process whereby a replica in a set of replicas becomes the new master for all of the other replicas. – Master M1 is replicating to replicas R1, R2 and R3. – Master M1 needs to be taken down for a hardware upgrade. – The DBA promotes R1 to be the master. – R2 and R3 are reconfigured & restarted, and now replicate from R1
  • 22.
    Streaming-only Remastering ● ● In priorversions, it wasn’t possible to stream from new master without taking a fresh backup That restriction is now gone
  • 23.
    Arch Independent Streaming ● StreamWAL from diferent OS and diferent architecture – pg_basebackup and pg_receivexlog can now work cross platform – You can’t apply WAL from diferent architecture, but this is still very useful for centralized backup etc
  • 24.
    Many More ● pg_basebackup confsetup (-R option) ● Logging enhancements ● Recursive views ● COPY PIPE ● Include_dir directive ● Array enhancements
  • 25.
  • 26.