question: take out the values of status and request_time in the log:
this log cannot be matched by script:
{"@timestamp":"2018-03-30T19:24:26+08:00","server_addr":"172.31.0.24","remote_addr":"10.59.23.86","scheme":"https","host":"api.mycomapp.com","method":"GET","uri":"/app/global/2/android.json?mark=gif&version=96&app=&language=en","url":"/app/global/2/android.json","protocol":"HTTP/1.1","status":"200","size":10206,"request_time":"0.159","upstream_time":"0.159","upstream_addr":"192.31.2.78:80","referer":"-","agent":"Mozilla/5.0 (Linux; Android 6.0.1; SM-G900M Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.111 Mobile Safari/537.36 News/96 Android/96 AppFootball/96 SDK/23 PackageName/com.mycom.news","http_x_forwarded_for":"-","uuid":"@hu87x0VcRPqJ9SwLZtQVpJWArHrnN7xA/j0cHVxLDY/ZUVIxXhZn","authorization":"-","lang":"en-US","route_id":"4c64bfcc996f11381e19d85b2e8ce9f3","product":"mycom","subsys":"api","uuidx":"-","version_name":"2.6.2","package":"com.mycom.news","auid":"-"}
the python code is as follows:
-sharp!/usr/bin/env python
import re
import sys
regex = re.compile(r"{".*"status":(\d+).*request_time":(\d+\.\d+)")
def process_line(line):
ro = regex.match(line)
if ro :
status , reqtime= ro.groups()
return status,reqtime
if __name__ == "__main__":
for line in sys.stdin:
print process_line(line)
execute as follows:
-sharp cat t.log |python logclean.py
None
logs that can be matched:
{"@timestamp":"2018-03-30T18:31:27+08:00","server_addr":"192.31.3.181","remote_addr":"197.210.173.163","scheme":"http","host":"api.abccccdapp.com","method":"GET","uri":"/app/archives/info?id=614464&language=en","url":"/index.php","protocol":"HTTP/1.0","status":200,"size":390,"request_time":0.006,"upstream_time":"0.006","upstream_host":"127.0.0.1:9000","referer":"abccccd://v1/main/home/tablist/http://app.abccccdapp.com/navite?push","agent":"Mozilla/5.0 (Linux; Android 7.1.1; SM-C7108 Build/NMF26X; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36 News/100 Android/100 App/100 SDK/25 PackageName/com.abccccd.news","xff":"197.21.173.163","uuid":"@nBFS5qbkHR3H0rJ2tFlR4cJsz","authorization":"-","lang":"en-US","route_id":"8c2352d0045f02c79722cb03261c0f88","sign":"MGD8xl2Ue6lagzzorNUnsCFNpsb7QcOVWf1QI="}
-sharp cat 2.txt|python logclean.py
("200", "0.006")
The regularities do not match. If you return None, how do you write the regularities? Thank you