Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Dec 16, 2024 - 08:00 UTC
Scheduled
As part of our ongoing effort to improve archive search performance and efficiency, we are introducing changes to the S3 bucket permissions needed for the logs/Traces S3 bucket. These changes will allow us to compact log and trace archive files, resulting faster query times and slightly smaller storage footprint. The update will take effect on December 16th, 2024.
Who is this relevant for? This change applies to any Coralogix customer that uses Coralogix’s archive feature for logs and/or traces stored in S3. Since this is one of the core components in Coralogix most customers will need to update the permissions.
What is happening? On December 16th, 2024, Coralogix will introduce a new compaction method for the archive files for Logs and Traces stored in S3 bucket. This means that files will be consolidated. Compaction will happen only once, on new files, every 20 minutes or once the total size of them reaches a size of 1GB (according to the earlier). Coralogix will need new permissions to enable file compaction.
What should you do? To benefit from these improvements, please update your S3 bucket permissions by December 16th, 2024. Add the following permissions to the S3 bucket: s3:DeleteObject, s3:DeleteObjectVersion, s3:DeleteObjectTagging, s3:DeleteObjectVersionTagging We’ve updated our CloudFormation templates and tutorials to assist you in making these changes.
Why do you need all those permissions? s3:DeleteObject – Allows deleting objects from the S3 bucket. s3:DeleteObjectVersion – Allows deleting all versions of an object when bucket versioning is enabled. Object cannot be deleted when they have versions. s3:DeleteObjectTagging – Allows deleting tags from an object (used by the Archive retention feature). Object cannot be deleted when they are tagged. s3:DeleteObjectVersionTagging – Enables deleting tags for a versioned object.
Why is this change being made? This update is designed to enhance both storage efficiency and system performance:
Important for ETL users: If you run ETL processes or any other scripts querying your S3 bucket, you will need to adapt the process to extract according to the following logic: If you granted Coralogix PutObjectTagging permissions: Extract only files with the following tag value: CX_COMPACTION_LEVEL:2. In case you did not grant PutObjectTagging permissions: You should refrain from querying files from the last 20 minutes as they will be compacted.
What happens if you don’t update? If the S3 permissions are not updated, we will implement a workaround that replaces compacted log files with zero-byte files in your bucket. Although this workaround reduces storage, it is less efficient and may incur minor costs related to PUT objects.