kafka输出器

功能描述

Kafka输出器能够将特定的日志信息输出到指定Kafka集群队列中。

OpenAPI配置日志

配置参数说明

参数说明是否必填默认值值可能性
name实例名string
driver驱动名"kafka_output"
description描述string
topic消息topicstring
addresskafka地址,多个地址用,分割"127.0.0.1:9092,127.0.0.2:9092"
timeout超时时间,单位为second10int
version使用的kafka版本,格式如:0.11.0.0,默认为最新稳定版"0.11.0.0"
partition_typepartition的选择方式,默认采用hash,选择hash时,若partition_key为空,则采用随机选择randomhash["random","robin","hash","manual"]
partitionpartitionType为manual时,该项指定分区号0int
partition_keypartitionType为hash时,该项指定hash值string, 形如$read_ip,具体可参考formatter
typeformatter的类型"line"["line","json"]
formatterformatter的输出内容object

注意

  • formatter的配置教程点此进行跳转

返回参数说明

参数名类型是否必含说明
idstring实例id
namestring实例名
driverstring驱动名
descriptionstring描述
professionstring模块名
createstring创建时间
updatestring更新时间
topicstring消息topic
addressstringkafka地址,多个地址用,分割
timeoutint超时时间,单位为second
versionstring使用的kafka版本,格式如:0.11.0.0,为空则默认为最新稳定版
partition_typestringpartition的选择方式,默认采用hash,选择hash时,若partition_key为空,则采用随机选择random
partitionintpartitionType为manual时,该项指定分区号
partition_keystringpartitionType为hash时,该项指定hash值
typestringformatter的类型
formatterobjectformatter的输出内容

创建文件输出器示例

curl -X POST  \
  'http://127.0.0.1:9400/api/output' \
  -H 'Content-Type:application/json' \
  -d '{
	"name": "demo_kafka",
	"driver": "kafka_output",
	"topic": "test",
	"address": "127.0.0.1:9092,127.0.0.2:9092,127.0.0.3:9092",
	"partition_type": "manual",
	"partition": 0,
	"type": "line",
	"formatter": {
		"fields": ["$request_id", "$request", "$status", "@time", "@proxy", "$response_time"],
		"time": ["$msec", "$time_iso8601", "$time_local"],
		"proxy": ["$proxy_uri", "$proxy_scheme", "$proxy_addr"]
	}
}'

返回结果示例

{
	"create": "2022-06-14 17:11:00",
	"driver": "kafka_output",
	"id": "demo_kafka@output",
	"name": "demo_kafka",
	"description": "",
	"profession": "output",
	"update": "2022-06-14 17:11:00",
	"topic": "test",
	"address": "127.0.0.1:9092,127.0.0.2:9092,127.0.0.3:9092",
	"partition_type": "manual",
	"partition": 0,
	"timeout": 0,
	"version": "",
	"partition_key": "",
	"type": "line",
	"formatter": {
		"fields": ["$request_id", "$request", "$status", "@time", "@proxy", "$response_time"],
		"proxy": ["$proxy_uri", "$proxy_scheme", "$proxy_addr"],
		"time": ["$msec", "$time_iso8601", "$time_local"]
	}
}

Kafka输出器使用

Kafka输出器可用于access-log插件的日志输出,点此跳转至access-log插件。