跳转至

istio 分布式追踪

istio版本1.23.2，k8s版本v1.30.2, 以下插件的安装不建议用于生产环境。

强烈建议先看一下这个文档： 分布式追踪概述

1、skywalking

1.1、参考

1.2、安装

上面的参考一定要看哦。

1.2.1、istio安装

istioctl install -y --set profile=default

1.2.2、配置链路追踪

istioctl operator init

采用此配置来安装 Istio 时，将使用 SkyWalking Agent 作为默认的追踪器， 链路数据会被发送到 SkyWalking 后端。defaultProviders.metrics.prometheus 不配置的话，kiali就会缺数据。 

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: mesh-default
  namespace: istio-system
spec:
  meshConfig:
    defaultProviders:
      metrics:
      - prometheus
      tracing:
      - "skywalking"
    enableTracing: true
    extensionProviders:
    - name: "skywalking"
      skywalking:
        service: tracing.istio-system.svc.cluster.local
        port: 11800

在默认的配置文件中，采样率为 1%， 使用 Telemetry API 将其提高到 100%： 

apiVersion: telemetry.istio.io/v1
kind: Telemetry
metadata:
  name: mesh-default
  namespace: istio-system
spec:
  tracing:
  - randomSamplingPercentage: 100.00

重启istio组件： 

kubectl delete pod -l istio=pilot -n istio-system
kubectl delete pod -l istio=ingressgateway -n istio-system
kubectl delete pod -l istio=egressgateway -n istio-system

kubectl apply -f samples/addons/prometheus.yaml
kubectl apply -f samples/addons/grafana.yaml
kubectl apply -f samples/addons/kiali.yaml

kubectl label namespace default istio-injection=enabled
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml

1.2.3、部署skywalking

kubectl apply -f samples/addons/extras/skywalking.yaml

istioctl dashboard kiali
istioctl dashboard skywalking

for i in $(seq 1 1000); do curl -s -o /dev/null "http://127.0.0.1/productpage"; done
1.2.3-1
trace

2、jaeger

2.1、安装

2.1.1、参考文档

jaeger集成

jaeger使用

2.1.2、自己安装

安装istio，刻意少了一个流程，没有指定jaeger-collector，直接这种安装就会出现jaeger中只有jaeger-all-in-one 

istioctl install -y --set profile=default --set meshConfig.defaultConfig.tracing.sampling=100 --set meshConfig.enableTracing=true

解决办法 

istioctl operator init

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    enableTracing: true
    defaultConfig:
      tracing:
        sampling: 50
        zipkin:
          address: jaeger-collector.istio-system.svc.cluster.local:9411

重启已经部署的需要用到istio的自己的服务(bookinfo)，还有istio、ingressgateway、egressgateway。不重启就不生效。
kubectl delete pod -l istio=pilot -n istio-system
kubectl delete pod -l istio=ingressgateway -n istio-system
kubectl delete pod -l istio=egressgateway -n istio-system

一劳永逸的办法 

istioctl install -y --set profile=default --set meshConfig.defaultConfig.tracing.sampling=100 --set meshConfig.defaultConfig.tracing.zipkin.address=jaeger-collector.istio-system.svc.cluster.local:9411 --set meshConfig.enableTracing=true


kubectl apply -f samples/addons/prometheus.yaml
kubectl apply -f samples/addons/grafana.yaml
kubectl apply -f samples/addons/jaeger.yaml
kubectl apply -f samples/addons/kiali.yaml

kubectl label namespace default istio-injection=enabled
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml

istioctl dashboard kiali
istioctl dashboard jaeger


直接执行访问jaeger-ui会出现好多服务
for i in $(seq 1 1000); do curl -s -o /dev/null "http://127.0.0.1/productpage"; done

2.1.2-1
主页
2.1.2-2
链路

3、自定义程序测试

上面测试的是官方提供的程序bookinfo，如果只是把本文档的1和2操作了一遍，只能证明可以初步的使用skywalking和jaeger，但是自己换个程序可能就没办法玩了，正常就会出现链路不够长，链路没有实现跨服务。

只要实现链路上下文传递，当代理发送 span 信息的时候，应用程序需要附加适当的 HTTP 请求头信息，这样才能够把多个 span 加到同一个调用链。

3-1
自定义程序的链路

3.1、fastapi示例

3.1.1、应用传递header头

翻阅了很多文档，发现最后还是官方案例的代码有用，官方是用的flask做的示例，我不喜欢，所以就用了fastapi。

utils.py, 我这是直接从productpage中搞过来的。 

from opentelemetry import trace
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.b3 import B3MultiFormat
from opentelemetry.sdk.trace import TracerProvider

propagator = B3MultiFormat()
set_global_textmap(B3MultiFormat())
provider = TracerProvider()
# Sets the global default tracer provider
trace.set_tracer_provider(provider)

tracer = trace.get_tracer(__name__)


def getForwardHeaders(request):
    headers = {}

    # x-b3-*** headers can be populated using the OpenTelemetry span
    ctx = propagator.extract(carrier={k.lower(): v for k, v in request.headers.items()})
    propagator.inject(headers, ctx)

    # We handle other (non x-b3-***) headers manually
    # if 'user' in session:
    #     headers['end-user'] = session['user']

    # Keep this in sync with the headers in details and reviews.
    incoming_headers = [
        # All applications should propagate x-request-id. This header is
        # included in access log statements and is used for consistent trace
        # sampling and log sampling decisions in Istio.
        'x-request-id',

        # Lightstep tracing header. Propagate this if you use lightstep tracing
        # in Istio (see
        # https://istio.io/latest/docs/tasks/observability/distributed-tracing/lightstep/)
        # Note: this should probably be changed to use B3 or W3C TRACE_CONTEXT.
        # Lightstep recommends using B3 or TRACE_CONTEXT and most application
        # libraries from lightstep do not support x-ot-span-context.
        'x-ot-span-context',

        # Datadog tracing header. Propagate these headers if you use Datadog
        # tracing.
        'x-datadog-trace-id',
        'x-datadog-parent-id',
        'x-datadog-sampling-priority',

        # W3C Trace Context. Compatible with OpenCensusAgent and Stackdriver Istio
        # configurations.
        'traceparent',
        'tracestate',

        # Cloud trace context. Compatible with OpenCensusAgent and Stackdriver Istio
        # configurations.
        'x-cloud-trace-context',

        # Grpc binary trace context. Compatible with OpenCensusAgent nad
        # Stackdriver Istio configurations.
        'grpc-trace-bin',

        # b3 trace headers. Compatible with Zipkin, OpenCensusAgent, and
        # Stackdriver Istio configurations.
        # This is handled by opentelemetry above
        # 'x-b3-traceid',
        # 'x-b3-spanid',
        # 'x-b3-parentspanid',
        # 'x-b3-sampled',
        # 'x-b3-flags',

        # SkyWalking trace headers.
        'sw8',

        # Application-specific headers to forward.
        'user-agent',

        # Context and session specific headers
        'cookie',
        'authorization',
        'jwt',
    ]
    # For Zipkin, always propagate b3 headers.
    # For Lightstep, always propagate the x-ot-span-context header.
    # For Datadog, propagate the corresponding datadog headers.
    # For OpenCensusAgent and Stackdriver configurations, you can choose any
    # set of compatible headers to propagate within your application. For
    # example, you can propagate b3 headers or W3C trace context headers with
    # the same result. This can also allow you to translate between context
    # propagation mechanisms between different applications.

    for ihdr in incoming_headers:
        val = request.headers.get(ihdr)
        if val is not None:
            headers[ihdr] = val

    return headers

main.py 

from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
FastAPIInstrumentor.instrument_app(app)

3.1.2、完整代码

完整代码

请求 http://myservice-a.com/api/v1/testA/
3.1.2-1
链路

4、插件域名访问

4.1、Grafana

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: grafana-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-grafana
      protocol: HTTP
    hosts:
    - "grafana.myserver.com"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: grafana-vs
  namespace: istio-system
spec:
  hosts:
  - "grafana.myserver.com"
  gateways:
  - grafana-gateway
  http:
  - route:
    - destination:
        host: grafana
        port:
          number: 3000
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: grafana
  namespace: istio-system
spec:
  host: grafana
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

4.2、Kiali

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: kiali-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-kiali
      protocol: HTTP
    hosts:
    - "kiali.myserver.com"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: kiali-vs
  namespace: istio-system
spec:
  hosts:
  - "kiali.myserver.com"
  gateways:
  - kiali-gateway
  http:
  - route:
    - destination:
        host: kiali
        port:
          number: 20001
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: kiali
  namespace: istio-system
spec:
  host: kiali
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

4.3、Prometheus

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: prometheus-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-prom
      protocol: HTTP
    hosts:
    - "prometheus.myserver.com"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: prometheus-vs
  namespace: istio-system
spec:
  hosts:
  - "prometheus.myserver.com"
  gateways:
  - prometheus-gateway
  http:
  - route:
    - destination:
        host: prometheus
        port:
          number: 9090
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: prometheus
  namespace: istio-system
spec:
  host: prometheus
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

4.4、trace

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: tracing-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-tracing
      protocol: HTTP
    hosts:
    - "tracing.myserver.com"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: tracing-vs
  namespace: istio-system
spec:
  hosts:
  - "tracing.myserver.com"
  gateways:
  - tracing-gateway
  http:
  - route:
    - destination:
        host: tracing
        port:
          number: 80
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: tracing
  namespace: istio-system
spec:
  host: tracing
  trafficPolicy:
    tls:
      mode: DISABLE
---
EOF

本文阅读量  次

评论