reyesj2
2eea671857
more precise wording in kafka annotation
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-20 16:16:55 -04:00
reyesj2
73acfbf864
update kafka annotations
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-20 16:02:45 -04:00
reyesj2
b7eebad2a5
Update Kafka self reset & add initial Kafka wrapper scripts to build out
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-12 11:01:40 -04:00
reyesj2
628893fd5b
remove redundant 'kafka_' from annotations & defaults
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-11 11:56:21 -04:00
reyesj2
ca7b89c308
Added Kafka reset to SOC UI. Incase of changing an active broker to a controller topics may become unavailable. Resolving this would require manual intervention. This option allows running a reset to start from a clean slate to then configure cluster to desired state before reenabling Kafka as global pipeline.
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-11 11:21:13 -04:00
reyesj2
284c1be85f
Update Kafka controller(s) via SOC UI
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-06-10 11:08:54 -04:00
reyesj2
d9ec556061
Update some annotations and defaults
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-05-29 16:41:02 -04:00
reyesj2
093cbc5ebc
Reconfigure Kafka defaults
...
- Set default number of partitions per topic -> 3. Helps ensure that out of the box we can take advantage of multi-node Kafka clusters via load balancing across atleast 3 brokers. Also multiple searchnodes will be able to pull from each topic. In this case 3 searchnodes (consumers) would be able to pull from all topics concurrently.
- Set default replication factor -> 2. This is the minimum value required for redundancy. Every partition will have 1 replica. In this case if we have 2 brokers each topic will have 3 partitions (load balanced across brokers) and each partition will have a replica on separate broker for redundancy
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-05-02 15:10:13 -04:00
reyesj2
11055b1d32
Rename kafkapass -> kafka_pass
...
Run so-kafka-clusterid within nodes.sls state so switchover is consistent
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-04-29 14:09:09 -04:00
reyesj2
086ebe1a7c
Split kafka defaults between broker / controller
...
Setup config.map.jinja to update broker / controller / combined node types
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-04-29 09:08:14 -04:00
reyesj2
5a401af1fd
Update kafka process_x_roles annotation
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-04-22 16:44:35 -04:00
reyesj2
4ac04a1a46
add kafkapass soc annotation
...
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com >
2024-04-18 16:46:36 -04:00
m0duspwnens
a54a72c269
move kafka_cluster_id to kafka:cluster_id
2024-04-12 11:19:20 -04:00
m0duspwnens
d3bd56b131
disable logstash and redis if kafka enabled
2024-04-10 14:13:27 -04:00
m0duspwnens
86b984001d
annotations and enable/disable from ui
2024-04-10 10:39:06 -04:00
