{"id":659,"date":"2017-07-25T18:24:37","date_gmt":"2017-07-25T09:24:37","guid":{"rendered":"http:\/\/stay-ko.be\/?p=659"},"modified":"2017-07-25T18:31:24","modified_gmt":"2017-07-25T09:31:24","slug":"windows-server-apache-logs-upload-to-amazon-s3-by-fluentd","status":"publish","type":"post","link":"http:\/\/stay-ko.be\/fluentd\/windows-server-apache-logs-upload-to-amazon-s3-by-fluentd\/","title":{"rendered":"Windows server \u306e apache \u30ed\u30b0\u3092 fluentd \u3067 S3 \u306b\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b"},"content":{"rendered":"
Windows \u306e Web\u30b5\u30fc\u30d0\u30fc\u3092AWS\u306b\u79fb\u884c\u3057\u305f\u30bf\u30a4\u30df\u30f3\u30b0\u3067\u3001\u3053\u308c\u307e\u3067\u30ed\u30b0\u30b5\u30fc\u30d0\u30fc\u306b\u96c6\u7d04\u3057\u3066\u3044\u305f\u5404\u30b5\u30fc\u30d0\u30fc\u306e\u30a2\u30af\u30bb\u30b9\u30ed\u30b0\u7b49\u3082\u3001Amazon S3 \u3092\u4fdd\u7ba1\u5148\u306b\u5909\u66f4\u3057\u3088\u3046\u3068\u3044\u3046\u3053\u3068\u306b\u306a\u3063\u305f\u3002\u5f93\u6765\u3001\u30d0\u30c3\u30c1\u30b9\u30af\u30ea\u30d7\u30c8\u306a\u3069\u3092\u99c6\u4f7f\u3057\u3066\u3001\u30ed\u30b0\u30b5\u30a4\u30af\u30eb\u3084\u3001\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3092\u884c\u3063\u3066\u3044\u305f\u304c\u3001\u3044\u307e\u3069\u304d\u3063\u307d\u304f Fluentd \u3067\u30b5\u30af\u30c3\u3068\u51fa\u6765\u306a\u3044\u304b\u306a\u3001\u3068\u601d\u3063\u3066\u691c\u8a3c\u3057\u305f\u3002<\/p>\n
\u3072\u3068\u307e\u305a\u3001Windows Server \u304b\u3089 fluentd \u3067 S3 \u306b\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3067\u304d\u308b\u3068\u3053\u308d\u307e\u3067\u3002\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u3084\u3001\u30d0\u30c3\u30d5\u30a1\u3001\u30c1\u30e3\u30f3\u30af\u306e\u691c\u8a0e\u306f\u3001\u3068\u308a\u3042\u3048\u305a\u5fd8\u308c\u308b\u3002<\/p>\n
\u516c\u5f0f\u30b5\u30a4\u30c8<\/a>\u304b\u3089\u3000td-agent-3.0.1-0-x64-beta2.msi \u3092\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3057\u3001\u30a4\u30f3\u30b9\u30c8\u30fc\u30e9\u3092\u8d77\u52d5\u3002\u4f55\u3082\u8003\u3048\u305a\u300c\u6b21\u3078\u300d\u3092\u9023\u6253\u3057\u3066\u304a\u3051\u3070\u5165\u308a\u307e\u3059\u3002<\/p>\n td-agent.conf \u306e\u4f5c\u6210<\/p>\n \n C:\u00a5opt\u00a5td-agent\u00a5conf\u00a5td-agent.conf\n<\/p><\/blockquote>\n [\u30b9\u30bf\u30fc\u30c8] – [Td-agent] – [Td-agent Command Prompt] \u3092\u958b\u3044\u3066\u3001\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3002<\/p>\n \n fluentd -c conf\\td-agent.conf\n<\/p><\/blockquote>\n \u5225\u306e Td-agent Command Prompt\u3092\u958b\u3044\u3066\u3001\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3002<\/p>\n \n C:\\opt\\td-agent>echo {“message”:”hello”} | fluent-cat test.event\n<\/p><\/blockquote>\n \u5148\u306e\u30b3\u30f3\u30bd\u30fc\u30eb\u306b<\/p>\n \n test.event: {“message”:”hello”}\n<\/p><\/blockquote>\n \u306e\u3088\u3046\u306a\u30e1\u30c3\u30bb\u30fc\u30b8\u304c\u51fa\u3066\u3044\u308c\u3070OK\u3002<\/p>\n Td-agent Command Prompt \u3067\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3002<\/p>\n \n fluentd –reg-winsvc i \u4f7f\u7528\u3059\u308b\u30d7\u30e9\u30b0\u30a4\u30f3\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u305f\u3081\u3001Td-agent Command Prompt \u304b\u3089\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3002<\/p>\n \n fluent-gem install fluent-plugin-s3 -v 1.0.0.rc2 –no-document td-agent.conf \u306b\u4ee5\u4e0b\u3092\u8ffd\u52a0<\/p>\n \u3053\u306e\u307e\u307e\u3060\u3068\u3001\u3044\u3064\u307e\u3067\u305f\u3063\u3066\u3082 S3 \u3078\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3055\u308c\u306a\u304b\u3063\u305f\u3002\u304a\u304b\u3057\u3044\u306a\u3001\u3001\u3068\u601d\u3063\u3066\u3001td-agent.log \u3092\u78ba\u8a8d\u3057\u305f\u3068\u308d\u3053\u3001S3 \u306e API call \u6642\u306b\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u30a8\u30e9\u30fc\u304c\u3067\u3066\u3044\u307e\u3057\u305f\u3002<\/p>\n \n unexpected error error_class=RuntimeError error=”can’t call S3 API. Please check your aws_key_id \/ aws_sec_key or s3_region configuration. error = # \u300cfluentd\u306e\u30d7\u30e9\u30b0\u30a4\u30f3\u3067certificate verify failed\u30a8\u30e9\u30fc\u304c\u767a\u751f\u3057\u305f\u8a71<\/a>\u300d\u306a\u3069\u3092\u53c2\u8003\u306b\u3059\u308b\u3068\u3001\u3069\u3046\u3084\u3089SSL\u8a3c\u660e\u66f8\u306e\u554f\u984c\u306e\u3088\u3046\u3067\u3059\u3002Td-agent Command Prompt \u3067\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3057\u3001\u8868\u793a\u3055\u308c\u305f\u30d1\u30b9\u306b\u8a3c\u660e\u66f8\u30d5\u30a1\u30a4\u30eb\u304c\u5b58\u5728\u3059\u308b\u304b\u78ba\u8a8d\u3059\u308b\u3002<\/p>\n \n C:\\opt\\td-agent>ruby -ropenssl -e ‘puts OpenSSL::X509::DEFAULT_CERT_FILE’ \u79c1\u306e\u5834\u5408\u3001ssl \u30d5\u30a9\u30eb\u30c0\u306b\u306f\u5b58\u5728\u3057\u306a\u304b\u3063\u305f\u304c\u3001c:\/opt\/td-agent\/embedded\/ssl\/certs\/cacert.pem \u304c\u3042\u3063\u305f\u306e\u3067\u3001\u30b3\u30ec\u3092\u30b3\u30d4\u30fc\u3057\u3066\u3001cert.pem \u30d5\u30a1\u30a4\u30eb\u3068\u3057\u3066\u8a72\u5f53\u306e\u30d1\u30b9\u306b\u4fdd\u5b58\u3059\u308b\u3053\u3068\u3067\u3001\u30a8\u30e9\u30fc\u306f\u89e3\u6d88\u3055\u308c\u307e\u3057\u305f\u3002\u3082\u3057\u3001certs \u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306b\u3082\u7121\u3044\u5834\u5408\u306f\u3001cURL\u304c\u63d0\u4f9b\u3057\u3066\u3044\u308b\u30eb\u30fc\u30c8CA\u8a3c\u660e\u66f8<\/a>\u3092\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3057\u3066\u8a66\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n [\u30b5\u30fc\u30d3\u30b9] – [Fluentd Windows Service] \u3092\u8d77\u52d5\u3059\u308b\u3002\u5bfe\u8c61\u3068\u306a\u308b\u30ed\u30b0\u306b\u8ffd\u8a18\u3055\u308c\u308b\u3088\u3046\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c\u3002\u3057\u3070\u3089\u304f\u3057\u305f\u5f8c\u3001AWS\u30b3\u30f3\u30bd\u30fc\u30eb\u304b\u3089 S3 \u30d0\u30b1\u30c3\u30c8\u3092\u78ba\u8a8d\u3057\u3066\u3001\u30d5\u30a1\u30a4\u30eb\u304c\u5b58\u5728\u3057\u3066\u3044\u308c\u3070\u3001\u3068\u308a\u3042\u3048\u305aS3\u3078\u306e\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u306f\u6b63\u5e38\u306b\u52d5\u4f5c\u3057\u3066\u3044\u308b\u3002 \u4eca\u56de\u306e td-agent.conf \u306e\u8a2d\u5b9a\u3067\u306f\u300110\u5206\u6bce\u306b S3 \u3078\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b\u8a2d\u5b9a\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002\u3072\u3068\u307e\u305a\u78ba\u8a8d\u304c\u51fa\u6765\u305f\u306e\u3067\u3001\u4e0d\u8981\u3067\u3042\u308c\u3070\u30b5\u30fc\u30d3\u30b9\u3092\u505c\u6b62\u3057\u3066\u304a\u304d\u307e\u3057\u3087\u3046\u3002\u6642\u9593\u306e\u3042\u308b\u3068\u304d\u306b\u3001td-agent.conf \u306e\u5404\u30d1\u30e9\u30e1\u30fc\u30bf\u306b\u3064\u3044\u3066\u78ba\u8a8d\u3057\u3066\u3044\u304d\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n","protected":false},"excerpt":{"rendered":" \u306f\u3058\u3081\u306b Windows \u306e Web\u30b5\u30fc\u30d0\u30fc\u3092AWS\u306b\u79fb\u884c\u3057\u305f\u30bf\u30a4\u30df\u30f3\u30b0\u3067\u3001\u3053\u308c… Continue Reading →<\/a><\/p>\n","protected":false},"author":2,"featured_media":686,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[36],"tags":[15,37,22],"amp_validity":null,"amp_enabled":true,"_links":{"self":[{"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/posts\/659"}],"collection":[{"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/comments?post=659"}],"version-history":[{"count":38,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/posts\/659\/revisions"}],"predecessor-version":[{"id":697,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/posts\/659\/revisions\/697"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/media\/686"}],"wp:attachment":[{"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/media?parent=659"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/categories?post=659"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/stay-ko.be\/wp-json\/wp\/v2\/tags?post=659"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
<source>\n @type forward\n<\/source>\n\n<match test.**>\n @type stdout\n<\/match><\/pre>\n
Windows \u30b5\u30fc\u30d3\u30b9\u3078\u767b\u9332<\/h3>\n
\n fluentd –reg-winsvc-fluentdopt ‘-c C:\/opt\/td-agent\/conf\/td-agent.conf -o C:\/opt\/td-agent\/td-agent.log’\n<\/p><\/blockquote>\nS3\u30d7\u30e9\u30b0\u30a4\u30f3\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/h2>\n
\n fluent-gem install fluent-plugin-forest\n<\/p><\/blockquote>\n<source>\n @type tail\n path C:\/Apache24\/logs\/access.log\n pos_file C:\/Apache24\/logs\/access.log.pos\n tag s3.apache.access\n format none\n<\/source>\n<match s3.**>\n @type s3\n aws_key_id ******* # IAM \u30a2\u30af\u30bb\u30b9\u30ad\u30fc\n aws_sec_key ******** # IAM \u30b7\u30fc\u30af\u30ec\u30c3\u30c8\u30ad\u30fc\n s3_bucket fluentd-demo-xxx # xxx \u306f\u4efb\u610f\u306e\u6570\u5b57\n s3_region ap-northeast-1\n s3_object_key_format %{path}%{time_slice}\/%{index}.%{file_extension}\n path logs\/\n time_slice_format %Y\/%m\/%d\n <buffer time>\n @type file\n path C:\/Apache24\/logs\/fluent\/s3\n timekey 360\n <\/buffer>\n<\/match><\/pre>\n
SSL\u8a3c\u660e\u66f8\u306e\u914d\u7f6e<\/h2>\n
\n \/opt\/td-agent\/embedded\/ssl\/cert.pem\n<\/p><\/blockquote>\nWindows \u30b5\u30fc\u30d3\u30b9\u306e\u8d77\u52d5<\/h2>\n
\n<\/p>\n