title: How do you send Fargate EKS Fluent Bit Logs from different services to separate Cloudwatch groups?
body: <p>I have followed <a href="https://aws.amazon.com/blogs/containers/fluent-bit-for-amazon-eks-on-aws-fargate-is-here/" rel="nofollow noreferrer">this guide</a> to configure Fluent Bit and Cloudwatch on my EKS cluster, but currently all of the logs go to one log group. I tried to follow a separate tutorial that used a kubernetes plugin for Fluent Bit to tag the services before the reached the [OUTPUT] configuration. This caused issues because Fargate EKS currently does not handle Fluent Bit [INPUT] configurations as per the <a href="https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html" rel="nofollow noreferrer">bottom of this doc</a>.</p>
  <p>Has anyone encountered this before? I'd like to split the logs up into separate services.</p>
  <p>Here is my current YAML file .. I added the parser and filter to see if I could gain any additional information to work with over on Cloudwatch.</p>
  <pre><code>kind: Namespace
  apiVersion: v1
  metadata:
    name: aws-observability
    labels:
      aws-observability: enabled
  ---
  kind: ConfigMap
  apiVersion: v1
  metadata:
    name: aws-logging
    namespace: aws-observability
  data:
    parsers.conf: |
      [PARSER]
          Name docker
          Format json
          Time_Key time
          Time_Format %Y-%m-%dT%H:%M:%S.%L
          Time_Keep On
          
    filters.conf: |
      [FILTER]
          Name kubernetes
          Match kube.*
          Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
          Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
          # Kube_Tag_Prefix kube.var.log.containers.
          Kube_URL https://kubernetes.default.svc:443
          Merge_Log On
          Merge_Log_Key log_processed
          Use_Kubelet true
          Buffer_Size 0
          Dummy_Meta true
    
    output.conf: |
      [OUTPUT]
          Name cloudwatch_logs
          Match   *
          region us-east-1
          log_group_name fluent-bit-cloudwatch2
          log_stream_prefix from-fluent-bit-
          auto_create_group On
  </code></pre>