RAID-0 (between the NVMe and EBS), but it seems to lock us to the slowest of the two storage mediums (i.e. We are afraid that the ephemeral nature on instance storage can kill it any moment (if AWS decides to kill the instance for whatever reason). The actual performance probably depends a lot on how Postgresql organizes write, but we did see real-world difference in performance between the two types. That's around x100 slower then NVMe (50 microseconds latency). When reading the EBS documentation it says on all volume types, even io2 that it has "single-digit millisecond latency". Thus, on random writes, it will defeat the purpose of NVMe. from what I read, real time synchornization (RAID) will limit writes to the throughput of the slower disk. Perform periodic backups by telling raid to sync up the EBS-half of the mirror that is usually detachedĭoes it save time compared on real-time synchronization? I.e. However, once this stage is finished, the load becomes almost exclusively reads, which probably depend more on IOPS then on latency. modify values in a table, then join two tables to produce a third one etc.).
We need to load and process large amount of data (hundreds of GB) in a very short time. The normal sync won't have to be the entire drive, only changed blocks, since LVM knows which blocks are dirty.Ī little bit of orchestration around quiescing the DB before you break the mirror, and keeping the right set of WAL logs around to allow recovery should do it. Once the NVME drive is synced up, you break the mirror in the other direction and now operate off NVME again.Tell raid/LVM to sync up the NVME drive.This gets you back online very quickly but with degraded performance. On failure, you reinit the mirror based on the EBS copy and replay the relevant WAL logs.Perform periodic backups by telling raid to sync up the EBS-half of the mirror that is usually detached.Have postgres send WAL logs to a EBS, potentially PIOPS, volume.Use LVM or mdadm in a mirror setup, most of the time the mirror is 'broken' ie nvme-only.> Once everything is build it's OK to work on EBS while we re-sync to the instance storage. Is it possible to get any guarantee from AWS about persisting instance store? we don't mind paying.It will slow us down, but at least we can keep working. Replication to Postgresql server on another instance using EBS.RAID-0 (between the NVMe and EBS), but it seems to lock us to the slowest of the two storage mediums (i.e.The volume is ~1TB, meaning that copying it from EBS (at 125MB/s) will take over 2 hours. Once everything is build it's OK to work on EBS while we re-sync to the instance storage.
We want to mirror it to a persistent storage so in such case we don't have to re-build everything. We use instance NVMe storage for I/O intensive operations (Postgresql with lots of data to process in very short time). If you're posting a technical query, please include the following details, so that we can help you more efficiently:ĭoes this sidebar need an addition or correction? Tell us here public IP addresses or hostnames, account numbers, email addresses) before posting! ✻ Smokey says: reflect heat by using light-colored materials on the exterior of buildings to fight climate change!
Note: ensure to redact or obfuscate all confidential or identifying information (eg. News, articles and tools covering Amazon Web Services (AWS), including S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, AWS-CDK, Route 53, CloudFront, Lambda, VPC, Cloudwatch, Glacier and more.
0 kommentar(er)
