Untitled :: Apache Cassandra 6 Docs

Preview | Unofficial | For review only

compaction = {compaction_map}

Defines the strategy for cleaning up data after writes.

Syntax uses a simple JSON format:

compaction = {
     'class' : '<compaction_strategy_name>',
     '<property_name>' : <value> [, ...] }

where the <compaction_strategy_name> is UnifiedCompactionStrategy, SizeTieredCompactionStrategy, TimeWindowCompactionStrategy, or LeveledCompactionStrategy.

Use only compaction implementations bundled with Apache Cassandra. See Compaction Strategies for more details.

Common properties

The following properties apply to all compaction strategies.

compaction = {
     'class' : 'compaction_strategy_name',
     'enabled' : (true | false),
     'log_all' : (true | false),
     'only_purge_repaired_tombstone' : (true | false),
     'tombstone_threshold' : <ratio>,
     'tombstone_compaction_interval' : <sec>,
     'unchecked_tombstone_compaction' : (true | false) }

enabled

Enable background compaction.

true runs minor compactions
false disables minor compactions

Use nodetool enableautocompaction to start running compactions.

+ Default: true

log_all

Activates advanced logging for the entire cluster.

Default: false

only_purge_repaired_tombstone

Enabling this property prevents data from resurrecting when repair is not run within the gc_grace_seconds. When its been a long time between repairs, the database keeps all tombstones.

true - Only allow tombstone purges on repaired SSTables.
false - Purge tombstones on SSTables during compaction even if the table has not been repaired.

Default: false

tombstone_threshold

The ratio of garbage-collectable tombstones to all contained columns. If the ratio exceeds this limit, compactions starts only on that table to purge the tombstones.

Default: 0.2

tombstone_compaction_interval

Number of seconds before compaction can run on an SSTable after it is created. An SSTable is eligible for compaction when it exceeds the tombstone_threshold. Because it might not be possible to drop tombstones when doing a single SSTable compaction, and since the compaction is triggered base on an estimated tombstone ratio, this setting makes the minimum interval between two single SSTable compactions tunable to prevent an SSTable from being constantly re-compacted.

Default: 86400 (1 day)

unchecked_tombstone_compaction

Setting to true allows tombstone compaction to run without pre-checking which tables are eligible for the operation. Even without this pre-check, Apache Cassandra checks an SSTable to make sure it is safe to drop tombstones.

Default: false

UnifiedCompactionStrategy

Unified Compaction Strategy is the recommended compaction strategy for most deployments. The following properties only apply to UnifiedCompactionStrategy:

compaction = {
     'class' : 'UnifiedCompactionStrategy',
     'scaling_parameters' : list of scaling parameters,
     'target_sstable_size': <text>,
     'base_shard_count': <int>,
     'parallelize_output_shards': (true | false) }

scaling_parameters

The maximum number of SSTables before a minor compaction is triggered.

Default: 32

target_sstable_size

The target size for SSTables that use the UnifiedCompactionStrategy. You must specify the bytes.

Default: 1 GiB

base_shard_count

The minimum number of shards, used for levels with the smallest density.

Default: 4 (1 for system tables, or when multiple data locations are defined)

expired_sstable_check_frequency_seconds

Determines how often to check for expired SSTables.

Default: 10 minutes

sstable_growth

The sstable growth component lambda, applied as a factor in the shard exponent calculation. This is a number between 0 and 1 that controls what part of the density growth should apply to individual sstable size and what part should increase the number of shards. Using a value of 1 has the effect of fixing the shard count to the base value. Using 0.5 makes the shard count and sstable size grow with the square root of the density growth. This is useful to decrease the sheer number of sstables that will be created for very large data sets. For example, without growth correction a data set of 10TiB with 1GiB target size would result in over 10k sstables, which may present as too much overhead both as on-heap memory used by per-sstable structures as well as time to look for intersecting sstables and tracking overlapping sets during compaction. Applying in this scenario (with base count 4) will reduce the potential number of sstables to ~160 of ~64GiB, which is still manageable both as memory overhead and individual compaction duration and space overhead. The balance between the two can be further tweaked by increasing to get fewer but bigger sstables on the top level, and decreasing it to favour a higher count of smaller sstables. The default value is 0.333 meaning the sstable size grows with the square root of the growth of the shard count.

parallelize_output_shards

When true, UCS splits individual compaction operations into separate parallel subtasks — one per output shard — and executes them concurrently. This reduces compaction wall-clock duration but disables preemptive SSTable opening for parallelized compactions.

The default can be overridden cluster-wide via the JVM system property -Dunified_compaction.parallelize_output_shards.

Example:

ALTER TABLE mykeyspace.foo
WITH compaction = {
     'class': 'UnifiedCompactionStrategy',
     'parallelize_output_shards': 'false' };

Default: true

SizeTieredCompactionStrategy

The compaction class SizeTieredCompactionStrategy (STCS) triggers a minor compaction when table meets the min_threshold. Minor compactions do not involve all the tables in a keyspace. See SizeTieredCompactionStrategy (STCS).

STCS is the default compaction strategy.

The following properties only apply to SizeTieredCompactionStrategy:

compaction = {
     'class' : 'SizeTieredCompactionStrategy',
     'bucket_high' : <factor>,
     'bucket_low' : <factor>,
     'min_sstable_size' : <int>,
     'min_threshold' : <int>,
     'max_threshold' : <int> }

min_threshold: The minimum number of SSTables to trigger a minor compaction.

Restriction: Not used in LeveledCompactionStrategy.

Default: 4
max_threshold: The maximum number of SSTables before a minor compaction is triggered.

Restriction: Not used in LeveledCompactionStrategy.

Default: 32
bucket_high: Size-tiered compaction merges sets of SSTables that are approximately the same size. The database compares each SSTable size to the average of all SSTable sizes for this table on the node. It merges SSTables whose size in KB are within [average-size * bucket_low] and [average-size * bucket_high].

Default: 1.5
bucket_low: Size-tiered compaction merges sets of SSTables that are approximately the same size. The database compares each SSTable size to the average of all SSTable sizes for this table on the node. It merges SSTables whose size in KB are within [average-size * bucket_low] and [average-size * bucket_high].

Default: 0.5
min_sstable_size: STCS groups SSTables into buckets. The bucketing process groups SSTables that differ in size by less than 50%. This bucketing process is too fine-grained for small SSTables. If your SSTables are small, use this option to define a size threshold in MB below which all SSTables belong to one unique bucket.

Default: 50 (MB)

The cold_reads_to_omit property for SizeTieredCompactionStrategy (STCS) is no longer supported.

LeveledCompactionStrategy

The compaction class LeveledCompactionStrategy (LCS) creates SSTables of a fixed, relatively small size (160 MB by default) that are grouped into levels. Within each level, SSTables are guaranteed to be non-overlapping. Each level (L0, L1, L2 and so on) is 10 times as large as the previous. Disk I/O is more uniform and predictable on higher than on lower levels as SSTables are continuously being compacted into progressively larger levels. At each level, row keys are merged into non-overlapping SSTables in the next level. See LeveledCompactionStrategy (LCS).

For more guidance, see When to Use Leveled Compaction and Leveled Compaction blog.

The following properties only apply to LeveledCompactionStrategy:

compaction = {
     'class' : 'LeveledCompactionStrategy,
     'sstable_size_in_mb' : <int> }

sstable_size_in_mb

The target size for SSTables that use the LeveledCompactionStrategy. Although SSTable sizes should be less or equal to sstable_size_in_mb, it is possible that compaction could produce a larger SSTable during compaction. This occurs when data for a given partition key is exceptionally large. The Apache Cassandra database does not split the data into two SSTables.

Default: 160

The default value, 160 MB, may be inefficient and negatively impact database indexing and the queries that rely on indexes. For example, consider the benefit of using higher values for sstable_size_in_mb in tables that use (SAI) indexes. For related information, see Compaction strategies.

fanout_size

The target size of levels increases by this fanout_size multiplier. You can reduce the space amplification by tuning this option. Default: 10

TimeWindowCompactionStrategy

The compaction class TimeWindowCompactionStrategy (TWCS) compacts SSTables using a series of time windows or buckets. TWCS creates a new time window within each successive time period. During the active time window, TWCS compacts all SSTables flushed from memory into larger SSTables using STCS. At the end of the time period, all of these SSTables are compacted into a single SSTable. Then the next time window starts and the process repeats. See TimeWindowCompactionStrategy (TWCS).

All of the properties for STCS are also valid for TWCS.

The following properties apply only to TimeWindowCompactionStrategy:

compaction = {
     'class' : 'TimeWindowCompactionStrategy,
     'compaction_window_unit' : <days>,
     'compaction_window_size' : <int> }

compaction_window_unit: Time unit used to define the bucket size. The value is based on the Java TimeUnit. For the list of valid values, see the Java API TimeUnit page located at https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/TimeUnit.html.

Default: days
compaction_window_size: Units per bucket.

Default: 1

DateTieredCompactionStrategy (deprecated)

Use TimeWindowCompactionStrategy instead.

Stores data written within a certain period of time in the same SSTable.

base_time_seconds: The size of the first time window.

Default: 3600
max_sstable_age_days (deprecated): Apache Cassandra does not compact SSTables if its most recent data is older than this property. Fractional days can be set.

Default: 1000
max_window_size_seconds: The maximum window size in seconds.

Default: 86400
timestamp_resolution: Units, <MICROSECONDS> or <MILLISECONDS>, to match the timestamp of inserted data.

Default: MICROSECONDS