By default, the retention duration is 7 days or 168 hours. After the retention duration is complete, the system purges the events. Based on the available disk space, the value of the log.retention.hours parameter can be set in the server.properties file at the $KAFKA_HOME/config/ location.
The sample performance runs with the following configuration:
Number of Kafka servers for high availability: 3
Number of Producers: 4 (CM, AM, CD, Event Aggregator)
Campaign Jobs: 200
Number of Agents: 1000
Contacts/Attempts: 12495000
Execution duration : 105 hours
Disk size of kafka-store directory: 45 GB
Number of Consumers(c): 5 (EventSDK sample client, Event Aggregator app)
Number of topics: 6
Replication factor(R): 3
Retention Period in Days (RP): 7
Total Attempts |
Expected Dialing Attempts Per Hour |
No. of Hours divided with attempts |
Total Size in GB |
Per Hour Size in MB |
12495000 |
119000 |
105 |
45 |
428.57 |
Topic |
MB/hour |
POM.Default.AGENT |
10.17 |
POM.Default.AGENTSTATISTICS |
1.04 |
POM.Default.ATTEMPT |
12.58 |
POM.Default.ENRICHEDATTEMPTRESULT |
7.49 |
POM.Default.JOB |
1.36 |
POM.Default.JOBSTATISTICS |
5.03 |
POM.Default.IMPORTSTATISTICS |
0.01 |
POM.Default.INBOUNDSKILL |
0.01 |
POM.HEARTBEAT |
2.32 |
Zookeeper directory size |
0.18 |
Total |
40.19 |
Based on this, you can calculate our cluster-wide disk size according to retention period.
MB or hour depends on the call flow, dialing parameters, and agents for the campaigns.
