Http 接口测试框架 (更新重试机制、清理测试数据、response 字段校验)

相关链接

关于这个框架

设计初衷:

解决我们项目的接口测试痛点。从之前的1-2小时测试时间压缩到现在的1分钟以内,效率提升,效果显著

对于读者:

  • 完全符合你们项目
    • 庆幸的是入手即用,方便快捷
    • 需要注意的是,不要做伸手党,可以了解这个框架后,再去针对性的优化,让它更符合你们项目
  • 部分符合你们项目
    • 提取出部分内容,加入到你们项目中
  • 完全不符合你们项目
    • 提供一种思路,虽然可能没什么用,但是能了解到别人对这件事是怎么思考的

关于框架代码结构:

  • 非程序出身,能写成这样我自己还是比较满意的
  • 结构设计可能有些问题,但是不影响使用

    • 如发现设计不合理之处,欢迎指正
    • 先出成果再作优化
  • 此框架服务于测试流程、效率,是一个工具

  • 至少目前认为手工+自动化才是最符合我们项目

本次更新

  • 新增重试机制
  • 清理测试数据
  • response字段校验
  • 稳定性提升
重试机制

主要用于重试连接超时接口、response响应码非200的接口、其他异常情况

  • 待全部接口遍历后,读取写入本地的数据,取出已遍历的接口名
  • 读取遍历前的全部接口名,与上一步的数据求diff

    • 可以是fiddler录制的接口数据
    • 可以是上一次接口回归后写入本地的数据
  • diff接口再从遍历前的接口中取出相关数据,加入重试队列

  • 重试上述步骤,直至diff不存在或重试次数耗尽

代码片段

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
<span class="k">def</span> <span class="nf">retry11</span><span class="p">(</span><span class="n">app_type</span><span class="p">,</span> <span class="n">retry</span><span class="o">=</span><span class="mi">3</span><span class="p">):</span>
    <span class="s">"""
    重试机制,默认3次
    :param app_type: 0 &gt;&gt; A; 1 &gt;&gt; B; 2 &gt;&gt; C; 3 &gt;&gt; D
    :param retry: 重试次数
    :return:
    """</span>
    <span class="n">r1</span> <span class="o">=</span> <span class="n">Retry</span><span class="p">(</span><span class="n">retry</span><span class="p">)</span>
    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">r1</span><span class="o">.</span><span class="n">get_diff</span><span class="p">())</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="s">'发现diff接口,重试机制启动...'</span><span class="p">)</span>
        <span class="n">r1</span><span class="o">.</span><span class="n">retry1</span><span class="p">(</span><span class="n">app_type</span><span class="p">)</span>


<span class="k">class</span> <span class="nc">Retry</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">retry</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">retry</span> <span class="o">=</span> <span class="n">retry</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">after_normal_sessions_path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span>
            <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">SESSIONS_PATH</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">Sessions</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">HOST</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_normal_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取遍历后正常(接口通过)的接口列表
        :return:
        """</span>
        <span class="k">return</span> <span class="n">utils</span><span class="o">.</span><span class="n">FileUtil</span><span class="o">.</span><span class="n">get_file_list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">after_normal_sessions_path</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_not_normal_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取需要人工验证的接口VerifyRequest
        :return:
        """</span>
        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_check_after_sessions</span><span class="p">(</span><span class="s">'VerifyRequest'</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_crash_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取程序异常接口ProgramCrash
        :return:
        """</span>
        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_check_after_sessions</span><span class="p">(</span><span class="s">'ProgramCrash'</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_unexpected_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取非预期接口Unexpected
        :return:
        """</span>
        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_check_after_sessions</span><span class="p">(</span><span class="s">'Unexpected'</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_field_change_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取字段改变接口FieldChange
        :return:
        """</span>
        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_check_after_sessions</span><span class="p">(</span><span class="s">'FieldChange'</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_check_after_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sessions_type</span><span class="p">):</span>
        <span class="s">"""
        获取需要检查的接口列表,已去重
        :param sessions_type:
        :return:
        """</span>
        <span class="n">path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">after_normal_sessions_path</span><span class="p">,</span> <span class="s">'</span><span class="se">\\</span><span class="s">Check</span><span class="se">\\</span><span class="s">'</span><span class="p">,</span> <span class="n">sessions_type</span><span class="p">,</span> <span class="s">'.txt'</span><span class="p">)</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">l</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="s">'utf-8'</span><span class="p">)</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span>
            <span class="n">sessions1</span> <span class="o">=</span> <span class="p">(</span><span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">,</span> <span class="s">''</span><span class="p">)[::</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="mi">1</span><span class="p">)[</span><span class="mi">0</span><span class="p">][::</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="s">'.txt'</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">l</span> <span class="k">if</span>
                         <span class="n">i</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'Request url: '</span><span class="p">))</span>
            <span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">sessions1</span><span class="p">))</span>
        <span class="k">except</span> <span class="n">FileNotFoundError</span><span class="p">:</span>
            <span class="k">return</span> <span class="p">()</span>

    <span class="k">def</span> <span class="nf">get_diff</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取diff接口
        diff 接口包含类型
        1.response 响应码 非200的接口
        2.请求超时的接口
        3.其他未知情况的接口(如 代码异常导致)
        :return:
        """</span>
        <span class="n">before_sessions</span> <span class="o">=</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">BEFORE_SESSIONS</span>
        <span class="n">after_sessions</span> <span class="o">=</span> <span class="p">[]</span>
        <span class="n">normal_after_sessions</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_normal_after_sessions</span><span class="p">()</span>
        <span class="n">not_normal_after_sessions</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_not_normal_after_sessions</span><span class="p">()</span>
        <span class="n">unexpected_after_sessions</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_unexpected_after_sessions</span><span class="p">()</span>
        <span class="n">field_change_after_sessions</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_field_change_after_sessions</span><span class="p">()</span>
        <span class="n">crash_after_sessions</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_crash_after_sessions</span><span class="p">()</span>
        <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">normal_after_sessions</span><span class="p">))</span> <span class="o">!=</span> <span class="s">"&lt;class 'NoneType'&gt;"</span><span class="p">:</span>
            <span class="n">after_sessions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">normal_after_sessions</span><span class="p">)</span>
        <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">not_normal_after_sessions</span><span class="p">))</span> <span class="o">!=</span> <span class="s">"&lt;class 'NoneType'&gt;"</span><span class="p">:</span>
            <span class="n">after_sessions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">not_normal_after_sessions</span><span class="p">)</span>
        <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">crash_after_sessions</span><span class="p">))</span> <span class="o">!=</span> <span class="s">"&lt;class 'NoneType'&gt;"</span><span class="p">:</span>
            <span class="n">after_sessions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">crash_after_sessions</span><span class="p">)</span>
        <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">unexpected_after_sessions</span><span class="p">))</span> <span class="o">!=</span> <span class="s">"&lt;class 'NoneType'&gt;"</span><span class="p">:</span>
            <span class="n">after_sessions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">unexpected_after_sessions</span><span class="p">)</span>
        <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">field_change_after_sessions</span><span class="p">))</span> <span class="o">!=</span> <span class="s">"&lt;class 'NoneType'&gt;"</span><span class="p">:</span>
            <span class="n">after_sessions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">field_change_after_sessions</span><span class="p">)</span>
        <span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">before_sessions</span><span class="p">)</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">after_sessions</span><span class="p">)))</span>

    <span class="k">def</span> <span class="nf">__get_diff_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        从本地磁盘读取diff接口sessions(request url; request header; ...)
        pass: 一个接口多条session
        :return:
        """</span>
        <span class="n">diff</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_diff</span><span class="p">()</span>
        <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">diff</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="s">'diff sessions: </span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="p">))</span>
            <span class="n">total_session</span> <span class="o">=</span> <span class="n">sessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="p">()</span><span class="o">.</span><span class="n">get_single_session</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
            <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">total_session</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
                <span class="k">print</span><span class="p">(</span><span class="s">'发现录制异常接口:'</span> <span class="o">+</span> <span class="n">d</span><span class="p">)</span>
                <span class="k">print</span><span class="p">(</span><span class="s">'执行移除操作,移除重试队列'</span><span class="p">)</span>
                <span class="c"># 移除录制异常的接口</span>
                <span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">SESSIONS_PATH</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">Api</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">HOST</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="n">d</span><span class="p">))</span>
                <span class="c"># 全局变量遍历前的全部接口也需要移除</span>
                <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">BEFORE_SESSIONS</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">yield</span> <span class="n">sessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="p">()</span><span class="o">.</span><span class="n">get_single_session</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__will_request_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        将要重跑的sessions,把多个接口的多个session合并为一个列表
        :return:
        """</span>
        <span class="n">s</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_diff_sessions</span><span class="p">()</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">s</span><span class="p">:</span>
            <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">i</span><span class="p">:</span>
                <span class="k">yield</span> <span class="n">j</span>

    <span class="k">def</span> <span class="nf">__request_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">app_type</span><span class="p">):</span>
        <span class="s">"""
        请求接口
        :return:
        """</span>
        <span class="n">s</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__will_request_sessions</span><span class="p">()</span>
        <span class="n">base</span><span class="o">.</span><span class="n">Request</span><span class="o">.</span><span class="n">thread_pool</span><span class="p">(</span><span class="n">app_type</span><span class="p">,</span> <span class="n">s</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">retry1</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">app_type</span><span class="p">):</span>
        <span class="s">"""
        重试的接口类型
        1.response 响应码 非200的接口
        2.请求超时的接口
        3.其他未知情况的接口(如 代码异常导致)
        注意:已经知道失败的接口不会再重试,目前是这样考虑的
        :return:
        """</span>
        <span class="n">temp</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">retry</span>
        <span class="k">while</span> <span class="bp">self</span><span class="o">.</span><span class="n">retry</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">retry</span> <span class="o">-=</span> <span class="mi">1</span>
            <span class="c"># 请求接口</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">__request_sessions</span><span class="p">(</span><span class="n">app_type</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="s">'第</span><span class="si">%</span><span class="s">d次尝试请求diff...'</span> <span class="o">%</span> <span class="p">(</span><span class="n">temp</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">retry</span><span class="p">,</span> <span class="p">))</span>
            <span class="c"># 再次求差异化文件,还有diff继续,否则停止</span>
            <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">get_diff</span><span class="p">())</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">retry</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
                <span class="k">print</span><span class="p">(</span><span class="s">'发现diff存在,继续尝试请求...'</span><span class="p">)</span>
                <span class="k">continue</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">break</span>
        <span class="k">print</span><span class="p">(</span><span class="s">'diff请求完成...'</span><span class="p">)</span>
清理测试数据

目的:在于避免接口回放创建的数据(如发朋友圈)对线上数据的影响

思路:通过一个接口对(如:发朋友圈与删除朋友圈),即创建数据与删除数据完成操作

  • 配置文件填写创建数据接口名以及response body json中创建数据成功的字段
  • 配置文件填写删除数据的接口名以及request body中传入删除数据的字段
  • 移除删除接口不加入执行接口列表
  • 执行接口请求,请求完毕后读取写入的创建接口数据文件,提取出字段
  • 执行接口后提取出删除接口的url request body response body存入list
  • 接口执行完毕时遍历创建数据的list,调用相应的删除接口删除数据
代码片段

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
<span class="k">def</span> <span class="nf">clear_up</span><span class="p">(</span><span class="n">app_type</span><span class="p">):</span>
    <span class="s">"""
    执行清理创建的数据
    :param app_type:
    :return:
    """</span>
    <span class="k">print</span><span class="p">(</span><span class="s">"清理创建的接口数据..."</span><span class="p">)</span>
    <span class="n">DelaySessions</span><span class="p">()</span><span class="o">.</span><span class="n">request_sessions</span><span class="p">(</span><span class="n">app_type</span><span class="p">)</span>


<span class="k">class</span> <span class="nc">DelaySessions</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">create_session_path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">SESSIONS_PATH</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">Sessions</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">HOST</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">delete_session_path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">SESSIONS_PATH</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">Api</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">HOST</span><span class="p">,</span> <span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_all_session_create_parameter</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">__get_single_session_create_parameter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">session_name</span><span class="p">):</span>
        <span class="s">"""
        获取单个创建数据接口response body json中创建数据成功的字段
        :return:
        """</span>
        <span class="n">total_session</span> <span class="o">=</span> <span class="p">[]</span>
        <span class="n">parameter</span> <span class="o">=</span> <span class="p">[]</span>
        <span class="n">file_path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">create_session_path</span><span class="p">,</span> <span class="n">session_name</span><span class="p">,</span> <span class="s">'.txt'</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">file_path</span><span class="p">):</span>
            <span class="n">total_session</span> <span class="o">=</span> <span class="n">sessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="p">()</span><span class="o">.</span><span class="n">get_single_session_full_path</span><span class="p">(</span>
                <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">create_session_path</span><span class="p">,</span> <span class="n">session_name</span><span class="p">,</span> <span class="s">'.txt'</span><span class="p">))</span>
        <span class="n">req</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="nb">compile</span><span class="p">(</span><span class="s">r'"</span><span class="si">%</span><span class="s">s":[0-9]+'</span> <span class="o">%</span> <span class="p">(</span><span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">CREATE_DICT</span><span class="p">[</span><span class="n">session_name</span><span class="p">],</span> <span class="p">))</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">total_session</span><span class="p">:</span>
            <span class="n">parameter</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">req</span><span class="p">,</span> <span class="n">i</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">":"</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
        <span class="k">return</span> <span class="n">parameter</span>

    <span class="k">def</span> <span class="nf">__get_all_session_create_parameter</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        获取所有创建数据接口response body json中创建数据成功的字段
        :return:
        """</span>
        <span class="n">create_parameter</span> <span class="o">=</span> <span class="p">{}</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">CREATE_DICT</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
            <span class="n">create_parameter</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_single_session_create_parameter</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">create_parameter</span>

    <span class="k">def</span> <span class="nf">__get_single_session_delete_parameter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">session_name</span><span class="p">):</span>
        <span class="s">"""
        替换单个删除数据接口request body中传入删除数据的字段值i
        :return:
        """</span>
        <span class="n">total_session</span> <span class="o">=</span> <span class="p">[]</span>
        <span class="n">delete_session_name</span> <span class="o">=</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">DELETE_DICT</span><span class="p">[</span><span class="n">session_name</span><span class="p">]</span>
        <span class="n">file_path</span> <span class="o">=</span> <span class="s">'</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">delete_session_path</span><span class="p">,</span> <span class="n">session_name</span><span class="p">,</span> <span class="s">'.txt'</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">file_path</span><span class="p">):</span>
            <span class="n">total_session</span> <span class="o">=</span> <span class="n">sessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="o">.</span><span class="n">ReadSessions</span><span class="p">()</span><span class="o">.</span><span class="n">get_single_session_full_path</span><span class="p">(</span><span class="n">file_path</span><span class="p">)</span>
        <span class="n">req</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="nb">compile</span><span class="p">(</span><span class="s">r'</span><span class="si">%</span><span class="s">s=[0-9]+'</span> <span class="o">%</span> <span class="p">(</span><span class="n">delete_session_name</span><span class="p">,</span> <span class="p">))</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">total_session</span><span class="p">:</span>
            <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span><span class="p">:</span>
                <span class="n">temp</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">req</span><span class="p">,</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">])[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'='</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
                <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">MAPPING_DICT</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
                    <span class="k">if</span> <span class="n">j</span> <span class="o">==</span> <span class="n">session_name</span><span class="p">:</span>
                        <span class="c"># 匹配对应的创建数据接口</span>
                        <span class="n">create_session_name</span> <span class="o">=</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">MAPPING_DICT</span><span class="p">[</span><span class="n">session_name</span><span class="p">]</span>
                        <span class="c"># 取第一个值,用完删除</span>
                        <span class="n">value</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span><span class="p">[</span><span class="n">create_session_name</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
                        <span class="n">i</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">i</span><span class="p">)</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">temp</span><span class="p">),</span> <span class="n">value</span><span class="p">)</span>  <span class="c"># 替换value</span>
                        <span class="n">l</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span><span class="p">[</span><span class="n">create_session_name</span><span class="p">])</span>
                        <span class="n">l</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
                        <span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span><span class="p">[</span><span class="n">create_session_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">l</span>
                        <span class="k">return</span> <span class="nb">eval</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">__get_all_session_delete_parameter</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="s">"""
        替换全部删除数据接口request body中传入删除数据的字段值,并返回一个即将请求接口的list
        :return:
        """</span>
        <span class="c"># 根据创建数据接口找到对应的删除接口,并拿出创建数据接口值的长度,多长就调用多少次对应删除接口</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
            <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">MAPPING_DICT</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
                <span class="k">if</span> <span class="n">utils</span><span class="o">.</span><span class="n">GlobalList</span><span class="o">.</span><span class="n">MAPPING_DICT</span><span class="p">[</span><span class="n">j</span><span class="p">]</span> <span class="o">==</span> <span class="n">i</span><span class="p">:</span>
                    <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">create_sessions_parameter_value</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">):</span>
                        <span class="c"># 此处可优化 目前会读取多次文件</span>
                        <span class="k">yield</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_single_session_delete_parameter</span><span class="p">(</span><span class="n">j</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">request_sessions</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">app_type</span><span class="p">):</span>
        <span class="s">"""
        请求删除接口
        :return:
        """</span>
        <span class="n">s</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__get_all_session_delete_parameter</span><span class="p">()</span>
        <span class="n">base</span><span class="o">.</span><span class="n">Request</span><span class="o">.</span><span class="n">thread_pool</span><span class="p">(</span><span class="n">app_type</span><span class="p">,</span> <span class="n">s</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="s">"接口数据清理完成!"</span><span class="p">)</span>
response字段校验

目的:校验是否缺失字段、校验字段类型是否改变、校验结果是否预期结果

思路:遍历response body json生成 <字段|字段类型的item> 存入list

  • 提取本地接口response body json存入list
  • 提取请求接口后的response body json存入list
  • 求diff list

pass:

  • 2个list可能存在字段相差太大的情况

    • 如一个接口本身返回了数据与无数据返回的情况
  • 解决方案

    • 求2个list的相似度
    • 100%相似则2个list长度相等用于验证字段类型改变
    • 相似度介于80%与100%之间(开区间)则判断是否缺失字段
    • 其他相似度则由于数据影响,没有太大比较意义,暂不考虑

收获:

字段类型改变(一般会导致客户端崩溃,当然客户端容错机制也没做到位)


1
2
3
Diff: ['CircleUserName|int', 'CircleUserName|str']
"CircleId":6420,"CircleUserName":0,"PostDate":""
"CircleId":6152,"CircleUserName":"虾米","PostDate":"2016年07月22日"

再细化的response字段校验

  • 目前的字段校验是抽象的,适用全部接口

  • 后面单个接口自动化时再针对单个接口的更细的字段校验

代码片段

遍历json取出字段及其类型


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
<span class="k">def</span> <span class="nf">decode_json</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">json_data</span><span class="p">):</span>
    <span class="s">"""
    解析json并返回对应的key|value
    :param json_data: json数据源
    :return: 返回json各字段以及字段值
    """</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="n">data</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">json_data</span><span class="p">)</span>
    <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="s">"JSON format error"</span><span class="p">)</span>
        <span class="k">return</span> <span class="p">[]</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">__iterate_json</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
    <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">json_list</span>


<span class="k">def</span> <span class="nf">__iterate_json</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">json_data</span><span class="p">,</span> <span class="n">i</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
    <span class="s">"""
    遍历json
    :param i: 遍历深度
    :param json_data: json数据源
    :return: 返回json各字段以及字段值
    """</span>
    <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">json_data</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
        <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">json_data</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">json_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s">'</span><span class="si">%</span><span class="s">s|</span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">]))</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">"'"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]))</span>
            <span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">]))</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"&lt;class 'list'&gt;"</span><span class="p">):</span>
                <span class="k">if</span> <span class="nb">len</span><span class="p">((</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">]))</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">dict</span><span class="p">):</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">__iterate_json</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="n">i</span><span class="o">=</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>
            <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">],</span> <span class="nb">dict</span><span class="p">):</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">__iterate_json</span><span class="p">(</span><span class="n">json_data</span><span class="p">[</span><span class="n">k</span><span class="p">],</span> <span class="n">i</span><span class="o">=</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="s">"JSON format error"</span><span class="p">)</span>

接口流程走向


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
接口回归测试启动...
清理测试数据...
读取配置文件中...
读取接口数据中...
接口请求中,请等待...
http://a-b.test.c.com/api/Circle/AddCancelCollectCircle
....................................................
http://a-b.test.c.com/api/GroupActivity/UploadActivityImage
http://a-b.test.c.com/api/photo/UploadImage
RequestException url: http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
HTTPConnectionPool(host='http://a-b.test.c.com', port=80): Read timed out. (read timeout=30)
IndexError url:
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
接口请求完成!
发现diff接口,重试机制启动...
第1次尝试请求diff...
diff sessions: GetDemandKnockSourceListV4.txt
diff sessions: GetSecondHouseTopic.txt
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
http://a-b.test.c.com/api/SecondHouseSource/GetSecondHouseTopic
RequestException url: http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
HTTPConnectionPool(host='http://a-b.test.c.com', port=80): Read timed out. (read timeout=30)
IndexError url:
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
发现diff存在,继续尝试请求...
第2次尝试请求diff...
diff sessions: GetDemandKnockSourceListV4.txt
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
RequestException url: http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
HTTPConnectionPool(host='http://a-b.test.c.com', port=80): Read timed out. (read timeout=30)
IndexError url:
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
发现diff存在,继续尝试请求...
第3次尝试请求diff...
diff sessions: GetDemandKnockSourceListV4.txt
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
RequestException url: http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
HTTPConnectionPool(host='http://a-b.test.c.com', port=80): Read timed out. (read timeout=30)
IndexError url:
http://a-b.test.c.com/api/Demand/GetDemandKnockSourceListV4
diff请求完成...
正在整理创建的数据...
清理创建的接口数据...
http://a-b.test.c.com/api/Circle/DeleteContent
http://a-b.test.c.com/api/Group/DeleteAnnouncement
http://a-b.test.c.com/api/RentDemand/DeleteRentDemandById
http://a-b.test.c.com/api/GroupFile/DeleteGroupFile
http://a-b.test.c.com/api/GroupDynamic/DeleteGroupDynamic
http://a-b.test.c.com/api/Demand/DeleteDemandById
http://a-b.test.c.com/api/GroupActivity/DeleteGroupActivity
接口数据清理完成!
测试报告准备中...
接口回归测试完成!
耗时: 125s

请求接口后写入本地的数据说明


1
2
3
4
5
FieldChange &gt;&gt; 字段改变的接口写入该文件
ProgramCrash &gt;&gt; 程序异常接口写入该文件
Unexpected &gt;&gt; 未达到预期字段校验的接口写入该文件
VerifyRequest &gt;&gt; 需要再次确认的接口写入该文件
GetUserInfoV2 &gt;&gt; 正常接口(一个接口一个文件)

关于接口回放的数据

  • 第一次的数据来自fiddler录制
  • 第二次及以后的数据来自第一次请求写入本地的正常接口文件 + fiddler继续录制需要检查的接口
  • 一般来说不要一直拿第一次fiddler录制的数据使用,目的在于测试接口对各个客户端版本的兼容情况
  • 接口回放可以跑线上运行的客户端全部版本接口(一个版本一套接口)

框架的下一步

  • 优雅的Html报告
  • 邮件通知
  • 持续集成
  • 测试数据另存,方便后续查阅

框架的更下一步

  • 接口压测
  • 接口自动化(api测试)pass:目前只是回归验证
  • 简单的GUI界面

GitHub

框架地址