from time import time # import xlrd import os # import shutil # import re # import sys # import json # import numpy as np # from pandas import DataFrame; # import pandas as pd import datetime from parsers.WriteToCsv import WriteBillToCsv class BillParser: #@构造函数 def __init__(self, time, path): self.m_time = time self.m_path = path self.m_logLine = {} def __GetValue(self, value_type, value): if value_type == "i" or value_type == "I": if value is None or value == "": return 0 else: return int(value) elif value_type == "s": if value is None or value == "": return "" else: #替换下逗号,存入csv return value.replace(",",";") def __ParserElement(self, element): components = element.split("=") if len(components) != 2: print(f"__ParserElement error {element}") return None, None value_type = components[0][0]; value = self.__GetValue(value_type, components[1]) key = components[0][1:] return key, value def __AddLog(self, log): type = log["Type"] if type in self.m_logLine.keys(): self.m_logLine[type].append(log) else: list = [] list.append(log) self.m_logLine[type] = list def __ParserLine(self, line): line = line.replace("\n", "") if len(line) < 20: return ret = {} elements = line.split("|") #每个元素按|分割 if elements.count == 0: return ret ret["logtime"] = elements.pop(0) #第一个元素是时间戳 try: timeArray = datetime.datetime.strptime(ret["logtime"], "%Y-%m-%d %H:%M:%S") except : print(f"__ParserLine logtime error {line}") return ret["timeStamp"] = int(timeArray.timestamp()) for element in elements: key, value = self. __ParserElement(element) if key == None or value == None: print(f"__ParserLine error {line}") continue ret[key] = value self.__AddLog(ret) #解析每个日志文件 def Parser(self, path, filename): print(f"Parser file {path}.........\n") #读取 file = open(path, "r", encoding="UTF-8") # line = file.readline() num = 0 while line: #逐行解析 #print(f"Parser file {filename} line:{line}\n") self.__ParserLine(line) line = file.readline() def __WriteCsv(self): date = self.m_time.strftime("%Y-%m-%d") for (inttype, logs) in self.m_logLine.items(): WriteBillToCsv(inttype, logs, date) def Parsers(self): date = self.m_time.strftime("%Y-%m-%d") fileRex = f"bill_{date}" list = os.listdir(self.m_path) for filename in list: path = os.path.join(self.m_path, filename) if os.path.isfile(path) and path.find(fileRex) != -1: self.Parser(path, filename) self.__WriteCsv()