I attended the Percona Live 2018 database conference in Santa Clara again.
As always, the conference was very well-organized and had great talks.
For 2018, Percona attempted to make it more affordable, with prices in the $600 range. Much appreciated!
Executive Summary
- MySQL 8 has been released with many new and improving features, including JSON and dynamic settings support
- PostgreSQL 10 has create partition and improving geo-distributed features
- ProxySQL can be used to replace HAProxy in HA database architectures.
I mainly attended the PostgreSQL track this year. The presenters were experts – either PostgreSQL server developers, professional trainers or AWS RDS/Aurora staff.
Tuesday Keynotes
Linux Performance 2018
Brendan Gregg, Netflix
Important to DBA
– BPF support added to perf
– BPF acts as a “sandbox” for monitoring agents
– Google’s BBR TCP algorithm 3x performance with 1% packet loss (hired Van Jacobsen)
– Facebook’s Kyber block reduce 99% latency by 300x
– Slides
Lectures
Real-time with Redis
Jon Hyman, Braze (formerly AppBoy)
– add jitter to timestamp to fix thundering herd
– 15 job queues
– SADD 7 GB arrays overwhelm replication slaves
– CPU utilization can stop replication
– API aggregation of keys
– fine-grained for user flexibility and triggered event use
– 120-140 redis servers at 40,000 ops/seconds
– expensive for replication
MySQL HA
– ProxySQL can help with 1 minute RDS failover and nearly-immediate Aurora failover
– ProxySQL does not forward the client IP yet (bug filed)
– ProxySQL is a good solution for filtering reads to slaves
Tuning Postgresql for High Write Workloads
Grant McAlister – Sr. Principal Engineer – Amazon RDS
– WAL compression will help reduce full page writes
– testing with random data may not provide useful results with WAL
– max_wal_size = 16GB, but increases recovery time
– checkpoint each minute
– async with random data maybe slower (code path not optimized for this workload)
– extra indexes in pg are expensive – hundreds or thousands of %
– prefix uuid with date to update btree on right (right lean)
– run vacuum to maintain perf (HOT and block cache) and avoid TID wraparound
– vacuum in memory:
– increase checkpoint_timeout
– alter table X set
– in memory before checkpoint 3.5 seconds
– In memory after checkpoint 84.5 seconds
– Vacuum not in memory 165.8 seconds
– Aurora: no checkpoints, no FWP, no log buffer
– not san, so block level. Also 4/6 quorum
– hash indexes are crash-safe in aurora 9.6, with be same with GA in 10
– managed service AWS staff gets paged on TID wrap-around
Securing Your Data on Postgresql
Payal Singh – OmniTI Computer Consulting Inc. payal@omniti.com
– Policies
– Row-level security (FORCE for table owner)
– Public is “public”
– before 10, policies were “OR”. 10 can do “AND”
– SSL (Certs needs to be 600 or restart will fail)
– event triggerimg like ddl event for auditing, can detect ownership changes
– table_rewrite notification to reject ddl and send message (possible solution for write amp)
– pg_audit
– at-rest encryption: pgcrypto. There is performance impact, so choose columns
– backups
– monitors to monitor queries for compliance (pg_stats extension)
– cryptdb can do encrypted queries (mysql-based)
– read-only replica can be a compliance problem because no logging possible
– desired features: redaction, oracle the, show grants
– pw strength is only on plaintext pa’s, not md5 or scram
– no rush for AWS KMS support
Saving Bandwidth When Using MySQL
Georgi Kodinov, Oracle MySQL SrvGen Team Lead (includes Security Team)
– Wireshark (smallest item is data response!) – disable SSL, will show sub requests
– SELECT and CALL generate metadata is large, but needed for MySQL client program, not for PHP apps. 443 vs. 106 bytes
– SET @@session.resultset_metadata=NONE;
– MySQL protocol has “capability flag” to help with backward compatibility
– Text protocol vs. binary protocol
– MySQL Protocol Doxygen in MySQL 8.0
Amazon Aurora MySQL and RDS MySQL: Lessons Learned
Mariella Di Giacomo
– did benchmarks with t2.medium and a large of RDS MySQL vs. Aurora MySQL. Aurora won, but I’m not sure of the difference in cost
Exhibit Hall
– got a demo of Vivid Cortex SAML/Okta support. Very nice UI.
VividCortex Dinner at Levi’s Stadium
– nice chance to meet other Vivid Cortex users and conference attendees
– Brendan Gregg is starting a cricket team at Netflix.
Wednesday Keynotes
Wednesday Lectures
PostgreSQL Replication
Christophe Pettus, PostgreSQL Experts thebuild.com
WAL-Based Replication
– first core replication was WAL shipping. All or nothing, monitor your disk. Same major version.
– archive_command can copy to secondary, manage it yourself
– for stream replication, use recovery.conf to point secondary at primary
– optional sync available (sometimes used to avoid lag)
– replicas can cascade
– max_standby_*_delay to control replication lag
. Use 0 for DR
– hot_standby_feedback
Trigger-Based Replication
– triggers cascade and run in alphabetical order
– on tables (Slony 1). Bucardo does multi-master replication (ping-pong protection)
– tedious, fiddly, performance impact, no DDL
– Slony requires C-language extensions, so cannot use on RDS
– Bucardo can be used on RDS
– RDS version 10 has WAL files access
Logical Decoding
– introduced in 9.4, improved in 9.6
– create replication slot to capture WAL stream
– tracks WAL position of consumer
– can run out of disk space
Replication Plugins
– can do whatever you want
– eg. export to Kafka
Logical Replication
– PG 10 has built-in logical replication, 9.4 has pg_logical
– pg_dump to dump schema
– primary key or unique index is a good idea, often required
– sequence values are not replicated, use disjoint ranges per server or UUIDs
– truncate not replicated, nor cascade
– replicate real table to real table, not view, FK
– pg 10 partitioning cannot be replicated because root table is not a real table
– no temp tables or unlogged tables
– COPY is individual sql inserts, could be millions
Pgpool2
– statement-based replication splits commands and sends to 2 servers – don’t use
Amazon DMS
– pg logical decoding
– timestamptz not supported at this time
2nd Qiuadrant BDR
– closed source, bi-directional
Streaming does DDL – reliable since 9.3, monitor secondary in case of quiet disconnect
PostgreSQL Replication
Simon Riggs, 2ndQuadrant
– Physical Streaming replication is sending WAL. Avail. 8 years. In-core
– Logical is non-WAL, like MySQL
– hot standby is same as read replica (AWS)
– repmgr and barman help mng
– postgres_fdw (foreign data wrapper)
– file_fdw access data like COPY
– PG pub/sub replication (push)
Multi-Node Advanced Features
– Push (Logical) or Pull Data Access (FDW)
– Multi-server hetero SQL
– Sharding (native in PG 11)
– Multi-node Query
– Multi-master database
– PostgreSQL-XL MPP similar to Teradata, Greenplum, Redshift
– Postgres-BDR Geo Cluster
– https://www.2ndquadrant.com/en/resources/postgres-bdr-2ndquadrant/
– BDR = Bidirectional Replication
Deep Dive into the RDS PostgreSQL Universe
AWS
– use cross-region read replica for migration
– create a paramater grep with rds.force_ssl=1
– RDS uses pg_upgrade or DMS
– OS Level Enhanced Monitoring
– Amazon RDS Performance Insights (Database Level) – Postgres Aurora today
Conference Closing Session
Lightning Talks
See this Percona Blog.
Prizes Giveaway
– Nice prizes for filling out the exhibitor pass.