• 行情收集器再升级–支持CSV格式文件导入提供自定义数据源

    最近一个用户需要让自己的CSV格式文件作为数据源,让发明者量化交易平台的回测系统使用。发明者量化交易平台的回测系统功能众多,使用简洁高效,这样只要自己有数据,就可以进行回测了,不再局限于平台数据中心支持的交易所、品种。

    设计思路

    设计思路其实很简单,我们只要在之前的行情收集器基础上稍微改动即可,我们给行情收集器增加一个参数

    isOnlySupportCSV

    用来控制是否只使用CSV文件作为数据源提供给回测系统,再增加一个参数

    filePathForCSV

    ,用于设置行情收集器机器人运行的服务器上放置CSV数据文件的路径。最后就是根据

    isOnlySupportCSV

    参数是否设置为

    True

    来决定使用那种数据源(1、自己收集的,2、CSV文件中的数据),这个改动主要在

    Provider

    类的

    do_GET

    函数中。

    什么是CSV文件?

    逗号分隔值(Comma-Separated Values,CSV,有时也称为字符分隔值,因为分隔字符也可以不是逗号),其文件以纯文本形式存储表格数据(数字和文本)。纯文本意味着该文件是一个字符序列,不含必须像二进制数字那样被解读的数据。CSV文件由任意数目的记录组成,记录间以某种换行符分隔;每条记录由字段组成,字段间的分隔符是其它字符或字符串,最常见的是逗号或制表符。通常,所有记录都有完全相同的字段序列。通常都是纯文本文件。建议使用WORDPAD或是记事本来开启,再则先另存新档后用EXCEL开启,也是方法之一。

    CSV文件格式的通用标准并不存在,但是有一定规律,一般为一条记录一行,第一行为表头。每行中的数据用逗号间隔。

    例如,我们用于测试的CSV文件用记事本打开是这样的:

    VPS教程

    观察下,CSV文件第一行是表格头。

    
    
    <span class="p">,</span><span class="nb">open</span><span class="p">,</span><span class="n">high</span><span class="p">,</span><span class="n">low</span><span class="p">,</span><span class="n">close</span><span class="p">,</span><span class="n">vol</span>

    我们就是要把这样的数据解析整理,然后构造成回测系统自定义数据源要求的格式,这个我们之前的文章中的代码里已经处理了,只需稍加修改。

    修改后的代码

    
    
    <span class="kn">import</span> <span class="nn">_thread</span>
    <span class="kn">import</span> <span class="nn">pymongo</span>
    <span class="kn">import</span> <span class="nn">json</span>
    <span class="kn">import</span> <span class="nn">math</span>
    <span class="kn">import</span> <span class="nn">csv</span>
    <span class="kn">from</span> <span class="nn">http.server</span> <span class="kn">import</span> <span class="n">HTTPServer</span><span class="p">,</span> <span class="n">BaseHTTPRequestHandler</span>
    <span class="kn">from</span> <span class="nn">urllib.parse</span> <span class="kn">import</span> <span class="n">parse_qs</span><span class="p">,</span> <span class="n">urlparse</span>

    <span class="k">def</span> <span class="nf">url2Dict</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
        <span class="n">query</span> <span class="o">=</span> <span class="n">urlparse</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">query</span>  
        <span class="n">params</span> <span class="o">=</span> <span class="n">parse_qs</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>  
        <span class="n">result</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="p">:</span> <span class="n">params</span><span class="p">[</span><span class="n">key</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">params</span><span class="p">}</span>  
        <span class="k">return</span> <span class="n">result</span>

    <span class="k">class</span> <span class="nc">Provider</span><span class="p">(</span><span class="n">BaseHTTPRequestHandler</span><span class="p">):</span>
        <span class="k">def</span> <span class="nf">do_GET</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
            <span class="k">global</span> <span class="n">isOnlySupportCSV</span><span class="p">,</span> <span class="n">filePathForCSV</span>
            <span class="k">try</span><span class="p">:</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">send_response</span><span class="p">(</span><span class="mi">200</span><span class="p">)</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">send_header</span><span class="p">(</span><span class="s2">&#034;Content-type&#034;</span><span class="p">,</span> <span class="s2">&#034;application/json&#034;</span><span class="p">)</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">end_headers</span><span class="p">()</span>

                <span class="n">dictParam</span> <span class="o">=</span> <span class="n">url2Dict</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">path</span><span class="p">)</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;自定义数据源服务接收到请求,self.path:&#034;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">path</span><span class="p">,</span> <span class="s2">&#034;query 参数:&#034;</span><span class="p">,</span> <span class="n">dictParam</span><span class="p">)</span>
               
                <span class="c1"># 目前回测系统只能从列表中选择交易所名称,在添加自定义数据源时,设置为币安,即:Binance</span>
                <span class="n">exName</span> <span class="o">=</span> <span class="n">exchange</span><span class="o">.</span><span class="n">GetName</span><span class="p">()</span>                                    
                <span class="c1"># 注意,period为底层K线周期</span>
                <span class="n">tabName</span> <span class="o">=</span> <span class="s2">&#034;</span><span class="si">%s</span><span class="s2">_</span><span class="si">%s</span><span class="s2">&#034;</span> <span class="o">%</span> <span class="p">(</span><span class="s2">&#034;records&#034;</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">dictParam</span><span class="p">[</span><span class="s2">&#034;period&#034;</span><span class="p">])</span> <span class="o">/</span> <span class="mi">1000</span><span class="p">))</span>  
                <span class="n">priceRatio</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="nb">pow</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">dictParam</span><span class="p">[</span><span class="s2">&#034;round&#034;</span><span class="p">]))</span>
                <span class="n">amountRatio</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="nb">pow</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">dictParam</span><span class="p">[</span><span class="s2">&#034;vround&#034;</span><span class="p">]))</span>
                <span class="n">fromTS</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">dictParam</span><span class="p">[</span><span class="s2">&#034;from&#034;</span><span class="p">])</span> <span class="o">*</span> <span class="nb">int</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span>
                <span class="n">toTS</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">dictParam</span><span class="p">[</span><span class="s2">&#034;to&#034;</span><span class="p">])</span> <span class="o">*</span> <span class="nb">int</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span>

                <span class="c1"># 要求应答的数据</span>
                <span class="n">data</span> <span class="o">=</span> <span class="p">{</span>
                    <span class="s2">&#034;schema&#034;</span> <span class="p">:</span> <span class="p">[</span><span class="s2">&#034;time&#034;</span><span class="p">,</span> <span class="s2">&#034;open&#034;</span><span class="p">,</span> <span class="s2">&#034;high&#034;</span><span class="p">,</span> <span class="s2">&#034;low&#034;</span><span class="p">,</span> <span class="s2">&#034;close&#034;</span><span class="p">,</span> <span class="s2">&#034;vol&#034;</span><span class="p">],</span>
                    <span class="s2">&#034;data&#034;</span> <span class="p">:</span> <span class="p">[]</span>
                <span class="p">}</span>
               
                <span class="k">if</span> <span class="n">isOnlySupportCSV</span><span class="p">:</span>
                    <span class="c1"># 处理CSV读取,filePathForCSV路径</span>
                    <span class="n">listDataSequence</span> <span class="o">=</span> <span class="p">[]</span>
                    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">filePathForCSV</span><span class="p">,</span> <span class="s2">&#034;r&#034;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
                        <span class="n">reader</span> <span class="o">=</span> <span class="n">csv</span><span class="o">.</span><span class="n">reader</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
                        <span class="c1"># 获取表头</span>
                        <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
                        <span class="n">headerIsNoneCount</span> <span class="o">=</span> <span class="mi">0</span>
                        <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">header</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s2">&#034;schema&#034;</span><span class="p">]):</span>
                            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;CSV文件格式有误,列数不同,请检查!&#034;</span><span class="p">,</span> <span class="s2">&#034;#FF0000&#034;</span><span class="p">)</span>
                            <span class="k">return</span>
                        <span class="k">for</span> <span class="n">ele</span> <span class="ow">in</span> <span class="n">header</span><span class="p">:</span>
                            <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s2">&#034;schema&#034;</span><span class="p">])):</span>
                                <span class="k">if</span> <span class="n">data</span><span class="p">[</span><span class="s2">&#034;schema&#034;</span><span class="p">][</span><span class="n">i</span><span class="p">]</span> <span class="o">==</span> <span class="n">ele</span> <span class="ow">or</span> <span class="n">ele</span> <span class="o">==</span> <span class="s2">&#034;&#034;</span><span class="p">:</span>
                                    <span class="k">if</span> <span class="n">ele</span> <span class="o">==</span> <span class="s2">&#034;&#034;</span><span class="p">:</span>
                                        <span class="n">headerIsNoneCount</span> <span class="o">+=</span> <span class="mi">1</span>
                                    <span class="k">if</span> <span class="n">headerIsNoneCount</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
                                        <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;CSV文件格式有误,请检查!&#034;</span><span class="p">,</span> <span class="s2">&#034;#FF0000&#034;</span><span class="p">)</span>
                                        <span class="k">return</span>
                                    <span class="n">listDataSequence</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
                                    <span class="k">break</span>
                       
                        <span class="c1"># 读取内容</span>
                        <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
                            <span class="n">record</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>
                            <span class="k">if</span> <span class="n">record</span> <span class="o">==</span> <span class="o">-</span><span class="mi">1</span><span class="p">:</span>
                                <span class="k">break</span>
                            <span class="n">index</span> <span class="o">=</span> <span class="mi">0</span>
                            <span class="n">arr</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span>
                            <span class="k">for</span> <span class="n">ele</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
                                <span class="n">arr</span><span class="p">[</span><span class="n">listDataSequence</span><span class="p">[</span><span class="n">index</span><span class="p">]]</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">ele</span><span class="p">)</span> <span class="k">if</span> <span class="n">listDataSequence</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">==</span> <span class="mi">0</span> <span class="k">else</span> <span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">ele</span><span class="p">)</span> <span class="o">*</span> <span class="n">amountRatio</span><span class="p">)</span> <span class="k">if</span> <span class="n">listDataSequence</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">==</span> <span class="mi">5</span> <span class="k">else</span> <span class="nb">int</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">ele</span><span class="p">)</span> <span class="o">*</span> <span class="n">priceRatio</span><span class="p">))</span>
                                <span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
                            <span class="n">data</span><span class="p">[</span><span class="s2">&#034;data&#034;</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">arr</span><span class="p">)</span>
                   
                    <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;数据:&#034;</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="s2">&#034;响应回测系统请求。&#034;</span><span class="p">)</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">wfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span><span class="o">.</span><span class="n">encode</span><span class="p">())</span>
                    <span class="k">return</span>
               
                <span class="c1"># 连接数据库</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;连接数据库服务,获取数据,数据库:&#034;</span><span class="p">,</span> <span class="n">exName</span><span class="p">,</span> <span class="s2">&#034;表:&#034;</span><span class="p">,</span> <span class="n">tabName</span><span class="p">)</span>
                <span class="n">myDBClient</span> <span class="o">=</span> <span class="n">pymongo</span><span class="o">.</span><span class="n">MongoClient</span><span class="p">(</span><span class="s2">&#034;mongodb://localhost:27017&#034;</span><span class="p">)</span>
                <span class="n">ex_DB</span> <span class="o">=</span> <span class="n">myDBClient</span><span class="p">[</span><span class="n">exName</span><span class="p">]</span>
                <span class="n">exRecords</span> <span class="o">=</span> <span class="n">ex_DB</span><span class="p">[</span><span class="n">tabName</span><span class="p">]</span>
               
                <span class="c1"># 构造查询条件:大于某个值{&#039;age&#039;: {&#039;$gt&#039;: 20}} 小于某个值{&#039;age&#039;: {&#039;$lt&#039;: 20}}</span>
                <span class="n">dbQuery</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#034;$and&#034;</span><span class="p">:[{</span><span class="s1">&#039;Time&#039;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#039;$gt&#039;</span><span class="p">:</span> <span class="n">fromTS</span><span class="p">}},</span> <span class="p">{</span><span class="s1">&#039;Time&#039;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#039;$lt&#039;</span><span class="p">:</span> <span class="n">toTS</span><span class="p">}}]}</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;查询条件:&#034;</span><span class="p">,</span> <span class="n">dbQuery</span><span class="p">,</span> <span class="s2">&#034;查询条数:&#034;</span><span class="p">,</span> <span class="n">exRecords</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">dbQuery</span><span class="p">)</span><span class="o">.</span><span class="n">count</span><span class="p">(),</span> <span class="s2">&#034;数据库总条数:&#034;</span><span class="p">,</span> <span class="n">exRecords</span><span class="o">.</span><span class="n">find</span><span class="p">()</span><span class="o">.</span><span class="n">count</span><span class="p">())</span>
               
                <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">exRecords</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">dbQuery</span><span class="p">)</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="s2">&#034;Time&#034;</span><span class="p">):</span>
                    <span class="c1"># 需要根据请求参数round和vround,处理数据精度</span>
                    <span class="n">bar</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;Time&#034;</span><span class="p">],</span> <span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;Open&#034;</span><span class="p">]</span> <span class="o">*</span> <span class="n">priceRatio</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;High&#034;</span><span class="p">]</span> <span class="o">*</span> <span class="n">priceRatio</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;Low&#034;</span><span class="p">]</span> <span class="o">*</span> <span class="n">priceRatio</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;Close&#034;</span><span class="p">]</span> <span class="o">*</span> <span class="n">priceRatio</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="s2">&#034;Volume&#034;</span><span class="p">]</span> <span class="o">*</span> <span class="n">amountRatio</span><span class="p">)]</span>
                    <span class="n">data</span><span class="p">[</span><span class="s2">&#034;data&#034;</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">bar</span><span class="p">)</span>
               
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;数据:&#034;</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="s2">&#034;响应回测系统请求。&#034;</span><span class="p">)</span>
                <span class="c1"># 写入数据应答</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">wfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span><span class="o">.</span><span class="n">encode</span><span class="p">())</span>
            <span class="k">except</span> <span class="ne">BaseException</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;Provider do_GET error, e:&#034;</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>


    <span class="k">def</span> <span class="nf">createServer</span><span class="p">(</span><span class="n">host</span><span class="p">):</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">server</span> <span class="o">=</span> <span class="n">HTTPServer</span><span class="p">(</span><span class="n">host</span><span class="p">,</span> <span class="n">Provider</span><span class="p">)</span>
            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;Starting server, listen at: </span><span class="si">%s</span><span class="s2">:</span><span class="si">%s</span><span class="s2">&#034;</span> <span class="o">%</span> <span class="n">host</span><span class="p">)</span>
            <span class="n">server</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
        <span class="k">except</span> <span class="ne">BaseException</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;createServer error, e:&#034;</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
            <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="s2">&#034;stop&#034;</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
        <span class="n">LogReset</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">isOnlySupportCSV</span><span class="p">):</span>
            <span class="k">try</span><span class="p">:</span>
            <span class="c1"># _thread.start_new_thread(createServer, ((&#034;localhost&#034;, 9090), ))         # 本机测试</span>
                <span class="n">_thread</span><span class="o">.</span><span class="n">start_new_thread</span><span class="p">(</span><span class="n">createServer</span><span class="p">,</span> <span class="p">((</span><span class="s2">&#034;0.0.0.0&#034;</span><span class="p">,</span> <span class="mi">9090</span><span class="p">),</span> <span class="p">))</span>         <span class="c1"># VPS服务器上测试</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;开启自定义数据源服务线程,数据由CSV文件提供。&#034;</span><span class="p">,</span> <span class="s2">&#034;#FF0000&#034;</span><span class="p">)</span>
            <span class="k">except</span> <span class="ne">BaseException</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;启动自定义数据源服务失败!&#034;</span><span class="p">)</span>
                <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;错误信息:&#034;</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
                <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="s2">&#034;stop&#034;</span><span class="p">)</span>
            <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
                <span class="n">LogStatus</span><span class="p">(</span><span class="n">_D</span><span class="p">(),</span> <span class="s2">&#034;只启动自定义数据源服务,不收集数据!&#034;</span><span class="p">)</span>
                <span class="n">Sleep</span><span class="p">(</span><span class="mi">2000</span><span class="p">)</span>
       
        <span class="n">exName</span> <span class="o">=</span> <span class="n">exchange</span><span class="o">.</span><span class="n">GetName</span><span class="p">()</span>
        <span class="n">period</span> <span class="o">=</span> <span class="n">exchange</span><span class="o">.</span><span class="n">GetPeriod</span><span class="p">()</span>
        <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;收集&#034;</span><span class="p">,</span> <span class="n">exName</span><span class="p">,</span> <span class="s2">&#034;交易所的K线数据,&#034;</span><span class="p">,</span> <span class="s2">&#034;K线周期:&#034;</span><span class="p">,</span> <span class="n">period</span><span class="p">,</span> <span class="s2">&#034;秒&#034;</span><span class="p">)</span>
       
        <span class="c1"># 连接数据库服务,服务地址 mongodb://127.0.0.1:27017 具体看服务器上安装的mongodb设置</span>
        <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;连接托管者所在设备mongodb服务,mongodb://localhost:27017&#034;</span><span class="p">)</span>
        <span class="n">myDBClient</span> <span class="o">=</span> <span class="n">pymongo</span><span class="o">.</span><span class="n">MongoClient</span><span class="p">(</span><span class="s2">&#034;mongodb://localhost:27017&#034;</span><span class="p">)</span>  
        <span class="c1"># 创建数据库</span>
        <span class="n">ex_DB</span> <span class="o">=</span> <span class="n">myDBClient</span><span class="p">[</span><span class="n">exName</span><span class="p">]</span>
       
        <span class="c1"># 打印目前数据库表</span>
        <span class="n">collist</span> <span class="o">=</span> <span class="n">ex_DB</span><span class="o">.</span><span class="n">list_collection_names</span><span class="p">()</span>
        <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;mongodb &#034;</span><span class="p">,</span> <span class="n">exName</span><span class="p">,</span> <span class="s2">&#034; collist:&#034;</span><span class="p">,</span> <span class="n">collist</span><span class="p">)</span>
       
        <span class="c1"># 检测是否删除表</span>
        <span class="n">arrDropNames</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">dropNames</span><span class="p">)</span>
        <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">arrDropNames</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
            <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">arrDropNames</span><span class="p">)):</span>
                <span class="n">dropName</span> <span class="o">=</span> <span class="n">arrDropNames</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
                <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">dropName</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
                    <span class="k">if</span> <span class="ow">not</span> <span class="n">dropName</span> <span class="ow">in</span> <span class="n">collist</span><span class="p">:</span>
                        <span class="k">continue</span>
                    <span class="n">tab</span> <span class="o">=</span> <span class="n">ex_DB</span><span class="p">[</span><span class="n">dropName</span><span class="p">]</span>
                    <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;dropName:&#034;</span><span class="p">,</span> <span class="n">dropName</span><span class="p">,</span> <span class="s2">&#034;删除:&#034;</span><span class="p">,</span> <span class="n">dropName</span><span class="p">)</span>
                    <span class="n">ret</span> <span class="o">=</span> <span class="n">tab</span><span class="o">.</span><span class="n">drop</span><span class="p">()</span>
                    <span class="n">collist</span> <span class="o">=</span> <span class="n">ex_DB</span><span class="o">.</span><span class="n">list_collection_names</span><span class="p">()</span>
                    <span class="k">if</span> <span class="n">dropName</span> <span class="ow">in</span> <span class="n">collist</span><span class="p">:</span>
                        <span class="n">Log</span><span class="p">(</span><span class="n">dropName</span><span class="p">,</span> <span class="s2">&#034;删除失败&#034;</span><span class="p">)</span>
                    <span class="k">else</span> <span class="p">:</span>
                        <span class="n">Log</span><span class="p">(</span><span class="n">dropName</span><span class="p">,</span> <span class="s2">&#034;删除成功&#034;</span><span class="p">)</span>
       
        <span class="c1"># 开启一个线程,提供自定义数据源服务</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="c1"># _thread.start_new_thread(createServer, ((&#034;localhost&#034;, 9090), ))     # 本机测试</span>
            <span class="n">_thread</span><span class="o">.</span><span class="n">start_new_thread</span><span class="p">(</span><span class="n">createServer</span><span class="p">,</span> <span class="p">((</span><span class="s2">&#034;0.0.0.0&#034;</span><span class="p">,</span> <span class="mi">9090</span><span class="p">),</span> <span class="p">))</span>         <span class="c1"># VPS服务器上测试</span>
            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;开启自定义数据源服务线程&#034;</span><span class="p">,</span> <span class="s2">&#034;#FF0000&#034;</span><span class="p">)</span>
        <span class="k">except</span> <span class="ne">BaseException</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;启动自定义数据源服务失败!&#034;</span><span class="p">)</span>
            <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;错误信息:&#034;</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
            <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="s2">&#034;stop&#034;</span><span class="p">)</span>
       
        <span class="c1"># 创建records表</span>
        <span class="n">ex_DB_Records</span> <span class="o">=</span> <span class="n">ex_DB</span><span class="p">[</span><span class="s2">&#034;</span><span class="si">%s</span><span class="s2">_</span><span class="si">%d</span><span class="s2">&#034;</span> <span class="o">%</span> <span class="p">(</span><span class="s2">&#034;records&#034;</span><span class="p">,</span> <span class="n">period</span><span class="p">)]</span>
        <span class="n">Log</span><span class="p">(</span><span class="s2">&#034;开始收集&#034;</span><span class="p">,</span> <span class="n">exName</span><span class="p">,</span> <span class="s2">&#034;K线数据&#034;</span><span class="p">,</span> <span class="s2">&#034;周期:&#034;</span><span class="p">,</span> <span class="n">period</span><span class="p">,</span> <span class="s2">&#034;打开(创建)数据库表:&#034;</span><span class="p">,</span> <span class="s2">&#034;</span><span class="si">%s</span><span class="s2">_</span><span class="si">%d</span><span class="s2">&#034;</span> <span class="o">%</span> <span class="p">(</span><span class="s2">&#034;records&#034;</span><span class="p">,</span> <span class="n">period</span><span class="p">),</span> <span class="s2">&#034;#FF0000&#034;</span><span class="p">)</span>
        <span class="n">preBarTime</span> <span class="o">=</span> <span class="mi">0</span>
        <span class="n">index</span> <span class="o">=</span> <span class="mi">1</span>
        <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
            <span class="n">r</span> <span class="o">=</span> <span class="n">_C</span><span class="p">(</span><span class="n">exchange</span><span class="o">.</span><span class="n">GetRecords</span><span class="p">)</span>
            <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">r</span><span class="p">)</span> <span class="o">&lt;</span> <span class="mi">2</span><span class="p">:</span>
                <span class="n">Sleep</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span>
                <span class="k">continue</span>
            <span class="k">if</span> <span class="n">preBarTime</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
                <span class="c1"># 首次写入所有BAR数据</span>
                <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">r</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">):</span>
                    <span class="n">bar</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
                    <span class="c1"># 逐根写入,需要判断当前数据库表中是否已经有该条数据,基于时间戳检测,如果有该条数据,则跳过,没有则写入</span>
                    <span class="n">retQuery</span> <span class="o">=</span> <span class="n">ex_DB_Records</span><span class="o">.</span><span class="n">find</span><span class="p">({</span><span class="s2">&#034;Time&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Time&#034;</span><span class="p">]})</span>
                    <span class="k">if</span> <span class="n">retQuery</span><span class="o">.</span><span class="n">count</span><span class="p">()</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
                        <span class="k">continue</span>
                   
                    <span class="c1"># 写入bar到数据库表</span>
                    <span class="n">ex_DB_Records</span><span class="o">.</span><span class="n">insert_one</span><span class="p">({</span><span class="s2">&#034;High&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;High&#034;</span><span class="p">],</span> <span class="s2">&#034;Low&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Low&#034;</span><span class="p">],</span> <span class="s2">&#034;Open&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Open&#034;</span><span class="p">],</span> <span class="s2">&#034;Close&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Close&#034;</span><span class="p">],</span> <span class="s2">&#034;Time&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Time&#034;</span><span class="p">],</span> <span class="s2">&#034;Volume&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Volume&#034;</span><span class="p">]})</span>                
                    <span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
                <span class="n">preBarTime</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s2">&#034;Time&#034;</span><span class="p">]</span>
            <span class="k">elif</span> <span class="n">preBarTime</span> <span class="o">!=</span> <span class="n">r</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s2">&#034;Time&#034;</span><span class="p">]:</span>
                <span class="n">bar</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span>
                <span class="c1"># 写入数据前检测,数据是否已经存在,基于时间戳检测</span>
                <span class="n">retQuery</span> <span class="o">=</span> <span class="n">ex_DB_Records</span><span class="o">.</span><span class="n">find</span><span class="p">({</span><span class="s2">&#034;Time&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Time&#034;</span><span class="p">]})</span>
                <span class="k">if</span> <span class="n">retQuery</span><span class="o">.</span><span class="n">count</span><span class="p">()</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
                    <span class="k">continue</span>
               
                <span class="n">ex_DB_Records</span><span class="o">.</span><span class="n">insert_one</span><span class="p">({</span><span class="s2">&#034;High&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;High&#034;</span><span class="p">],</span> <span class="s2">&#034;Low&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Low&#034;</span><span class="p">],</span> <span class="s2">&#034;Open&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Open&#034;</span><span class="p">],</span> <span class="s2">&#034;Close&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Close&#034;</span><span class="p">],</span> <span class="s2">&#034;Time&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Time&#034;</span><span class="p">],</span> <span class="s2">&#034;Volume&#034;</span><span class="p">:</span> <span class="n">bar</span><span class="p">[</span><span class="s2">&#034;Volume&#034;</span><span class="p">]})</span>
                <span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
                <span class="n">preBarTime</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s2">&#034;Time&#034;</span><span class="p">]</span>
            <span class="n">LogStatus</span><span class="p">(</span><span class="n">_D</span><span class="p">(),</span> <span class="s2">&#034;preBarTime:&#034;</span><span class="p">,</span> <span class="n">preBarTime</span><span class="p">,</span> <span class="s2">&#034;_D(preBarTime):&#034;</span><span class="p">,</span> <span class="n">_D</span><span class="p">(</span><span class="n">preBarTime</span><span class="o">/</span><span class="mi">1000</span><span class="p">),</span> <span class="s2">&#034;index:&#034;</span><span class="p">,</span> <span class="n">index</span><span class="p">)</span>
            <span class="c1"># 增加画图展示</span>
            <span class="n">ext</span><span class="o">.</span><span class="n">PlotRecords</span><span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="s2">&#034;</span><span class="si">%s</span><span class="s2">_</span><span class="si">%d</span><span class="s2">&#034;</span> <span class="o">%</span> <span class="p">(</span><span class="s2">&#034;records&#034;</span><span class="p">,</span> <span class="n">period</span><span class="p">))</span>
            <span class="n">Sleep</span><span class="p">(</span><span class="mi">10000</span><span class="p">)</span>

    运行测试

    首先我们启动行情收集器机器人,我们给机器人添加一个交易所,让机器人运行起来。
    参数配置:

    VPS教程
    VPS教程

    然后我们创建一个测试策略:

    
    
    <span class="kd">function</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
        <span class="nx">Log</span><span class="p">(</span><span class="nx">exchange</span><span class="p">.</span><span class="nx">GetRecords</span><span class="p">())</span>
        <span class="nx">Log</span><span class="p">(</span><span class="nx">exchange</span><span class="p">.</span><span class="nx">GetRecords</span><span class="p">())</span>
        <span class="nx">Log</span><span class="p">(</span><span class="nx">exchange</span><span class="p">.</span><span class="nx">GetRecords</span><span class="p">())</span>
    <span class="p">}</span>

    策略很简单,只获取并打印三次K线数据。

    回测页面,设置回测系统的数据源为自定义数据源,并且地址填写行情收集器机器人运行的服务器地址。由于我们的CSV文件中的数据为1分钟K线。所以回测时,我们设置K线周期为1分钟。

    VPS教程

    点击开始回测,行情收集器机器人接收到了数据请求:

    VPS教程

    回测系统执行策略完成后,根据数据源中的K线数据,生成K线图表。

  • + 亚马逊账户被关联的后果是什么?
  • + 第一次使用Linux,能给一点建议吗?
  • + 从PHP 5到PHP 7性能全评测(含未发布的JIT版PHP 8对比)
  • + 能开发出网页版的tiktok吗,不必安装app,数据都在国内,可以做到全球访问吗?
  • + 想租一个我的世界服务器,不知道哪些商家靠谱?
  • + 公司内网的服务器如何用远程桌面连接?
  • + 高2了学习不太好相当黑客给点建议吧?
  • + 如何使用国内服务器搭建论坛规避备案?
  • + 怎么靠亚马逊测评发家致富?
  • + 有什么酷炫的电脑开机方式?
  • + 又卡二审了 这些知识点你掌握了吗?
  • + 免费vnc,5款免费的绿色版vnc软件

评论

你必须先登录