Home avatar

蜷缩的蜗牛

专注云原生运维

Loki配置解析

Loki 可配置内容较多,相对复杂。本文通过分析 Loki 默认配置文件来了解一下。

默认配置

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
# A comma-separated list of components to run. The default value 'all' runs Loki
# in single binary mode. The value 'read' is an alias to run only read-path
# related components such as the querier and query-frontend, but all in the same
# process. The value 'write' is an alias to run only write-path related
# components such as the distributor and compactor, but all in the same process.
# Supported values: all, compactor, distributor, ingester, querier,
# query-scheduler, ingester-querier, query-frontend, index-gateway, ruler,
# table-manager, read, write. A full list of available targets can be printed
# when running Loki with the '-list-targets' command line flag.
# CLI flag: -target
[target: <string> | default = "all"]

# Enables authentication through the X-Scope-OrgID header, which must be present
# if true. If false, the OrgID will always be set to 'fake'.
# CLI flag: -auth.enabled
[auth_enabled: <boolean> | default = true]

# The amount of virtual memory in bytes to reserve as ballast in order to
# optimize garbage collection. Larger ballasts result in fewer garbage
# collection passes, reducing CPU overhead at the cost of heap size. The ballast
# will not consume physical memory, because it is never read from. It will,
# however, distort metrics, because it is counted as live memory.
# CLI flag: -config.ballast-bytes
[ballast_bytes: <int> | default = 0]

# Configures the server of the launched module(s).
[server: <server>]

# Configures the distributor.
[distributor: <distributor>]

# Configures the querier. Only appropriate when running all modules or just the
# querier.
[querier: <querier>]

# The query_scheduler block configures the Loki query scheduler. When configured
# it separates the tenant query queues from the query-frontend.
[query_scheduler: <query_scheduler>]

# The frontend block configures the Loki query-frontend.
[frontend: <frontend>]

# The query_range block configures the query splitting and caching in the Loki
# query-frontend.
[query_range: <query_range>]

# The ruler block configures the Loki ruler.
[ruler: <ruler>]

# The ingester_client block configures how the distributor will connect to
# ingesters. Only appropriate when running all components, the distributor, or
# the querier.
[ingester_client: <ingester_client>]

# The ingester block configures the ingester and how the ingester will register
# itself to a key value store.
[ingester: <ingester>]

# The index_gateway block configures the Loki index gateway server, responsible
# for serving index queries without the need to constantly interact with the
# object store.
[index_gateway: <index_gateway>]

# The storage_config block configures one of many possible stores for both the
# index and chunks. Which configuration to be picked should be defined in
# schema_config block.
[storage_config: <storage_config>]

# The chunk_store_config block configures how chunks will be cached and how long
# to wait before saving them to the backing store.
[chunk_store_config: <chunk_store_config>]

# Configures the chunk index schema and where it is stored.
[schema_config: <schema_config>]

# The compactor block configures the compactor component, which compacts index
# shards for performance.
[compactor: <compactor>]

# The limits_config block configures global and per-tenant limits in Loki.
[limits_config: <limits_config>]

# The frontend_worker configures the worker - running within the Loki querier -
# picking up and executing queries enqueued by the query-frontend.
[frontend_worker: <frontend_worker>]

# The table_manager block configures the table manager for retention.
[table_manager: <table_manager>]

# Configuration for memberlist client. Only applies if the selected kvstore is
# memberlist.
# 
# When a memberlist config with atleast 1 join_members is defined, kvstore of
# type memberlist is automatically selected for all the components that require
# a ring unless otherwise specified in the component's configuration section.
[memberlist: <memberlist>]

# Configuration for 'runtime config' module, responsible for reloading runtime
# configuration file.
[runtime_config: <runtime_config>]

# Configuration for tracing.
[tracing: <tracing>]

# Configuration for usage report.
[analytics: <analytics>]

# Common configuration to be shared between multiple modules. If a more specific
# configuration is given in other sections, the related configuration within
# this section will be ignored.
[common: <common>]

# How long to wait between SIGTERM and shutdown. After receiving SIGTERM, Loki
# will report 503 Service Unavailable status via /ready endpoint.
# CLI flag: -shutdown-delay
[shutdown_delay: <duration> | default = 0s]

target

Loki系统架构中提到 Loki 是微服务架构,它所有组件都是可以独立运行。Loki 内部针对这些组件定义一组别名,可以通过这个参

使用Shell实现Operaror

shell-operator架构

Shell Operator 是个冷僻又有点用的东西。方便运维监听kubernetes事件,并基于这些事件做一些简单任务处理;并且shell语言基本上大部分运维人员都懂,而不需要太高的学习成本。

运行原理

shell-operator 部署在 Pod 中。在 Pod 中有一个 /hooks 的一个子目录,其中存储了可执行文件,它们可以用 Bash、Python、Ruby等编写的,我们称这些可执行文件为hooks。Shell-opeator 订阅 Kubernetes 事件并执行这些钩子来响应我们感兴趣的事件。

源码分析 Kubernetes Scheduler 核心调度器的实现原理

https://xiaorui-cc.oss-cn-hangzhou.aliyuncs.com/images/202301/202301251502086.png

基于 kubernetes v1.27.0 源码分析 scheduler 调度器

k8s scheduler 的主要职责是为新创建的 pod 寻找一个最合适的 node 节点, 然后进行 bind node 绑定, 后面 kubelet 才会监听到并创建真正的 pod.

那么问题来了, 如何为 pod 寻找最合适的 node ? 调度器需要经过 predicates 预选和 priority 优选.

0%