PlayingWithKafka
所属分类:中间件编程
开发工具:C#
文件大小:0KB
下载次数:0
上传日期:2018-07-06 14:38:10
上 传 者:
sh-1993
说明: 在Windows Docker上玩Kafka
(Playing with Kafka on Windows Docker)
文件列表:
Avrogen/ (0, 2018-07-06)
Avrogen/Confluent.Apache.Avro.dll (124928, 2018-07-06)
Avrogen/Newtonsoft.Json.dll (468480, 2018-07-06)
Avrogen/System.CodeDom.dll (187224, 2018-07-06)
Avrogen/avrogen.deps.json (63788, 2018-07-06)
Avrogen/avrogen.dll (6144, 2018-07-06)
Avrogen/avrogen.runtimeconfig.json (146, 2018-07-06)
Avrogen/log4net.dll (221184, 2018-07-06)
Kafka/ (0, 2018-07-06)
Kafka/FaultyOrganisation.asvc (282, 2018-07-06)
Kafka/Infrastructure/ (0, 2018-07-06)
Kafka/Infrastructure/Configuration/ (0, 2018-07-06)
Kafka/Infrastructure/Configuration/KafkaConfiguration.cs (1116, 2018-07-06)
Kafka/Kafka.csproj (308, 2018-07-06)
Kafka/Kafka/ (0, 2018-07-06)
Kafka/Kafka/FaultyOrganisation.cs (2278, 2018-07-06)
Kafka/Kafka/Organisation.cs (2260, 2018-07-06)
Kafka/Organisation.asvc (276, 2018-07-06)
KafkaConsumer/ (0, 2018-07-06)
KafkaConsumer/.vscode/ (0, 2018-07-06)
KafkaConsumer/.vscode/launch.json (1209, 2018-07-06)
KafkaConsumer/.vscode/tasks.json (325, 2018-07-06)
KafkaConsumer/Application.cs (709, 2018-07-06)
KafkaConsumer/Infrastructure/ (0, 2018-07-06)
KafkaConsumer/Infrastructure/Configuration/ (0, 2018-07-06)
KafkaConsumer/Infrastructure/Configuration/KafkaConsumerConfiguration.cs (227, 2018-07-06)
KafkaConsumer/KafkaConsumer.csproj (1151, 2018-07-06)
KafkaConsumer/Program.cs (1625, 2018-07-06)
KafkaConsumer/Services/ (0, 2018-07-06)
KafkaConsumer/Services/IKafkaConsumerService.cs (118, 2018-07-06)
KafkaConsumer/Services/KafkaConsumerService.cs (2799, 2018-07-06)
KafkaConsumer/appsettings.json (20, 2018-07-06)
KafkaProducer/ (0, 2018-07-06)
KafkaProducer/.vscode/ (0, 2018-07-06)
KafkaProducer/.vscode/launch.json (1209, 2018-07-06)
KafkaProducer/.vscode/tasks.json (325, 2018-07-06)
KafkaProducer/Application.cs (709, 2018-07-06)
KafkaProducer/Infrastructure/ (0, 2018-07-06)
KafkaProducer/Infrastructure/Configuration/ (0, 2018-07-06)
... ...
# Playing with Kafka
## Setting up Kafka with Docker
### Prerequisites
* Docker for Windows - https://store.docker.com/editions/community/docker-ce-desktop-windows
* Windows Subsystem for Linux - https://docs.microsoft.com/en-us/windows/wsl/install-win10
* Docker on Windows Subsystem for Linux (Ubuntu) - https://medium.com/@sebagomez/installing-the-docker-client-on-ubuntus-windows-subsystem-for-linux-612b392a44c4
### Setting up Kafka
* https://hub.docker.com/r/confluent/kafka/
#### Stopping and removing containers
```bash
docker stop zookeeper | xargs docker rm
docker stop kafka | xargs docker rm
docker stop schema-registry | xargs docker rm
docker stop rest-proxy | xargs docker rm
docker stop schema-registry-ui | xargs docker rm
```
#### Start Zookeeper and expose port 2181 for use by the host machine
```bash
docker run -d --name zookeeper -p 2181:2181 confluent/zookeeper
```
#### Start Kafka and expose port 9092 for use by the host machine
*Also configure the broker to use the docker machine's IP address*
```bash
docker run -d --name kafka -p 9092:9092 --link zookeeper:zookeeper --env KAFKA_ADVERTISED_HOST_NAME=[DOCKER_HOST_IP] confluent/kafka
```
#### Start Schema Registry and expose port 8081 for use by the host machine
```bash
docker run -d --name schema-registry -p 8081:8081 --link zookeeper:zookeeper --link kafka:kafka confluent/schema-registry
```
#### Start REST Proxy and expose port 8082 for use by the host machine
```bash
docker run -d --name rest-proxy -p 8082:8082 --link zookeeper:zookeeper --link kafka:kafka --link schema-registry:schema-registry confluent/rest-proxy
```
#### Start Schema Registry UI and expose port 8000 for use by the host machine
```bash
docker run -d --name schema-registry-ui -p 8000:8000 -e "SCHEMAREGISTRY_URL=http://localhost:8081" landoop/schema-registry-ui
```
#### Start Kafka topics UI and expose port 8080 for use by the host machine
```bash
docker run -d -p 8080:8000 -e "KAFKA_REST_PROXY_URL=http://localhost:8082" -e "PROXY=true" landoop/kafka-topics-ui
```
*Don't forget to enable cross-origin resource sharing.*
### All-in-one solution `Lenses`
```bash
docker run -e ADV_HOST=127.0.0.1 -e EULA="https://dl.lenses.stream/d/?id=[LICENSEKEY]" --rm -p 3030:3030 -p 9092:9092 -p 2181:2181 -p 8081:8081 -p 9581:9581 -p 9582:9582 -p 9584:9584 -p 9585:9585 landoop/kafka-lenses-dev
```
### Generate C# classes from AVRO schemas
#### Avrogen
https://github.com/confluentinc/avro/releases/download/v1.7.7.4/avrogen.zip
```bash
C:\PROJECTDATA\PLAYGROUND\PlayingWithKafka\Kafka (master)
λ dotnet ..\Avrogen\avrogen.dll -s Organisation.asvc .
```
近期下载者:
相关文件:
收藏者: