cold's worldhttps://www.linuxzen.com/2021-12-21T00:00:00+08:00纸上得来终觉浅,绝知此事要躬行Rust actix-web Extractors 研究2021-12-21T00:00:00+08:002021-12-21T00:00:00+08:00coldtag:www.linuxzen.com,2021-12-21:/rust-actix-web-extractors.html<p>参见 <a href="https://www.linuxzen.com/notes/articles/20211221174651-actix_web_extractor/">actix-web extractors 支持提取任意长度的参数</a>。</p>【译】深入理解 Rust future2021-07-29T00:00:00+08:002021-07-29T00:00:00+08:00coldtag:www.linuxzen.com,2021-07-29:/understanding-rust-futures-by-going-way-too-deep.html<p>原文链接:<a href="https://fasterthanli.me/articles/understanding-rust-futures-by-going-way-too-deep">Understanding Rust futures by going way too deep</a>。</p>
<p>译者注:原文大量的引入了有趣的对话,迫于排版问题这里不进行翻译,必要的对话通过引用 …</p><p>原文链接:<a href="https://fasterthanli.me/articles/understanding-rust-futures-by-going-way-too-deep">Understanding Rust futures by going way too deep</a>。</p>
<p>译者注:原文大量的引入了有趣的对话,迫于排版问题这里不进行翻译,必要的对话通过引用块来解释。</p>
<h2 id="深入理解-rust-future">深入理解 Rust future</h2>
<p>用 Rust future!就是这么简单!直到我们发现并非如此。所以我们先探索简单的部分,然后继续探索困难部分而不是等它慢慢靠近我们。</p>
<h2 id="起步">起步</h2>
<blockquote>
<p>Choo choo here comes the easy part 🚂💨</p>
</blockquote>
<p>我们创建一个新的项目:</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">waytoodeep</span>
<span class="w"> </span><span class="n">Created</span><span class="w"> </span><span class="kt">binary</span><span class="w"> </span><span class="p">(</span><span class="n">application</span><span class="p">)</span><span class="w"> </span><span class="n n-Quoted">`waytoodeep`</span><span class="w"> </span><span class="n">package</span>
</code></pre></div>
<p>我们需要安装 <code>cargo-edit</code> 如果之前没有安装过的话,接下来就可以直接 <code>cargo add</code> :</p>
<div class="highlight"><pre><span></span><code><span class="o">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">install</span><span class="w"> </span><span class="n">cargo</span><span class="o">-</span><span class="n">edit</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="n">crates</span><span class="o">.</span><span class="n">io</span><span class="w"> </span><span class="n">index</span>
<span class="w"> </span><span class="n">Downloaded</span><span class="w"> </span><span class="n">cargo</span><span class="o">-</span><span class="n">edit</span><span class="w"> </span><span class="n">v0</span><span class="o">.</span><span class="mf">7.0</span>
<span class="w"> </span><span class="n">Downloaded</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">crate</span><span class="w"> </span><span class="p">(</span><span class="mf">57.6</span><span class="w"> </span><span class="n">KB</span><span class="p">)</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="mf">0.47</span><span class="n">s</span>
<span class="w"> </span><span class="n">Ignored</span><span class="w"> </span><span class="n">package</span><span class="w"> </span><span class="err">`</span><span class="n">cargo</span><span class="o">-</span><span class="n">edit</span><span class="w"> </span><span class="n">v0</span><span class="o">.</span><span class="mf">7.0</span><span class="err">`</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">already</span><span class="w"> </span><span class="n">installed</span><span class="p">,</span><span class="w"> </span><span class="n">use</span><span class="w"> </span><span class="o">--</span><span class="n">force</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">override</span>
</code></pre></div>
<blockquote>
<p>因为 <code>cargo-edit</code> 很方便,所以你可能已经安装过它。部分读者会感到困惑是因为像
<code>cargo new</code>, <code>cargo build</code>, <code>cargo test</code>, <code>cargo run</code> 等子命令都内置在 cargo 中,
但是 <code>cargo add</code> 没有。</p>
<p>实际上,有一大堆像这样的包,如 <a href="https://lib.rs/crates/cargo-hack">cargo-hack</a>,<a href="https://lib.rs/crates/cargo-udeps">cargo-udeps</a>,<a href="https://lib.rs/crates/cargo-expand">cargo-expand</a>...<a href="https://lib.rs/keywords/cargo">等等</a>。</p>
</blockquote>
<p>然后我们需要选择一个「异步运行时」(async runtime),因为这些 future 对象不会轮询(poll)自己。。。
我们毫无理由的选择 tokio,唯一的原因是:过去几个月我一直在用它。</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">tokio</span><span class="mf">@1.9.0</span><span class="w"> </span><span class="o">--</span><span class="n">features</span><span class="w"> </span><span class="n">full</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">tokio</span><span class="w"> </span><span class="n">v1</span><span class="mf">.9.0</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span><span class="w"> </span><span class="n">with</span><span class="w"> </span><span class="n">features</span><span class="o">:</span><span class="w"> </span><span class="p">[</span><span class="s">"full"</span><span class="p">]</span>
</code></pre></div>
<p>然后我们修改 <code>main</code> 函数使用 tokio 默认执行器(executor)( <code>cargo new</code> 为我们生成了一个 <code>main</code> 函数,但是这里并不能满足我们的需求):</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/main.rs`</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"Hello from a (so far completely unnecessary) async runtime"</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>3s<span class="w"> </span>209ms
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/Users/wh/codes/rust/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">3</span>.47s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Hello<span class="w"> </span>from<span class="w"> </span>a<span class="w"> </span><span class="o">(</span>so<span class="w"> </span>far<span class="w"> </span>completely<span class="w"> </span>unnecessary<span class="o">)</span><span class="w"> </span>async<span class="w"> </span>runtime
</code></pre></div>
<p>酷!</p>
<p>接下来让我们添加其他一些我喜欢在我的项目中使用的好东西。</p>
<p>首先,对于错误处理 - 我们编写程序就需要处理一堆不同库里不同的错误类型,如果能通过一个类型统一它们就会非常整洁。</p>
<p><a href="https://lib.rs/crates/eyre">eyre</a> 可以赋予我们这些(就像 <code>anyhow</code> )!</p>
<p>并且因为我喜欢漂亮的颜色我将使用 <a href="https://lib.rs/crates/color-eyre">color-eyre</a>。</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">color</span><span class="o">-</span><span class="n">eyre</span><span class="mf">@0.5.11</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">color</span><span class="o">-</span><span class="n">eyre</span><span class="w"> </span><span class="n">v0</span><span class="mf">.5.11</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
</code></pre></div>
<p>现在我们需要安装 <code>color-eyre</code> 作为默认的崩溃(panic)处理器,我悄悄修改了一些环境变量来默认输出调用堆栈(backtracks)。</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">color_eyre</span>::<span class="n">Report</span><span class="p">;</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"Hello from a (so far completely unnecessary) async runtime"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">setup</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">,</span><span class="w"> </span><span class="s">"1"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">color_eyre</span>::<span class="n">install</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.02s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Hello<span class="w"> </span>from<span class="w"> </span>a<span class="w"> </span><span class="o">(</span>so<span class="w"> </span>far<span class="w"> </span>completely<span class="w"> </span>unnecessary<span class="o">)</span><span class="w"> </span>async<span class="w"> </span>runtime
</code></pre></div>
<p>很好!现在如果我们某处出现了一个错误,我们将看到完整的堆栈跟踪,就像下面这样:
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/color-eyre.78931d5fc80841f6.webp"></p>
<p>最后,因为我喜欢结构化日志,让我们添加 <a href="https://lib.rs/crates/tracing">tracing</a> 然后通过漂亮的颜色打印它们,让我们添加 <a href="https://lib.rs/crates/tracing-subscriber">tracing-subscriber</a>.</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">tracing</span><span class="mf">@0.1.26</span><span class="w"> </span><span class="n">tracing</span><span class="o">-</span><span class="n">subscriber</span><span class="mf">@0.2.19</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">tracing</span><span class="w"> </span><span class="n">v0</span><span class="mf">.1.26</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">tracing</span><span class="o">-</span><span class="n">subscriber</span><span class="w"> </span><span class="n">v0</span><span class="mf">.2.19</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
</code></pre></div>
<p>我们已经有一个 <code>setup</code> 函数,所以直接在那里安装 <code>tracing-subscriber</code>.. 然后我们将 <code>println!</code> 改成 <code>info!</code> !
然后,为了演示如何设置让我们再次修改一些环境变量:对所有包(crates)默认 <code>info</code> 日志级别。</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">color_eyre</span>::<span class="n">Report</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing</span>::<span class="n">info</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">EnvFilter</span><span class="p">;</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello from a comfy nest we've made for ourselves"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">setup</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">,</span><span class="w"> </span><span class="s">"1"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">color_eyre</span>::<span class="n">install</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">,</span><span class="w"> </span><span class="s">"info"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">fmt</span>::<span class="n">fmt</span><span class="p">()</span>
<span class="w"> </span><span class="p">.</span><span class="n">with_env_filter</span><span class="p">(</span><span class="n">EnvFilter</span>::<span class="n">from_default_env</span><span class="p">())</span>
<span class="w"> </span><span class="p">.</span><span class="n">init</span><span class="p">();</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.02s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:03:46.993<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Hello<span class="w"> </span>from<span class="w"> </span>a<span class="w"> </span>comfy<span class="w"> </span>nest<span class="w"> </span>we<span class="err">'</span>ve<span class="w"> </span>made<span class="w"> </span><span class="k">for</span><span class="w"> </span>ourselves
</code></pre></div>
<p>好了,我们准备好做一些有用的事情了。</p>
<h3 id="做一些有用的事情">做一些有用的事情</h3>
<p>当决定在咖啡间隙阅读哪一篇文章的时候,人们通常同时打开几个网站,然后读最先加载出来的那一篇。</p>
<p>事实如此。你可以引用我的话,谁会去验证呢?毕竟这听起来需要很多工作。</p>
<p>所以让我们来编写一个程序做相同的事情。</p>
<p>让我们引入 <a href="https://lib.rs/crates/reqwest">reqwest</a> -- 尽管我不喜欢它的 API,但它会很好的完成接下来的工作。</p>
<p>同时,因为 <a href="https://www.openssl.org/news/vulnerabilities.html">screw OpenSSL</a> 我们将标记 reqwest 使用 <a href="https://lib.rs/crates/rustls">rustls</a>:</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">reqwest</span><span class="mf">@0.11.4</span><span class="w"> </span><span class="o">--</span><span class="n">no</span><span class="o">-</span><span class="k">default</span><span class="o">-</span><span class="n">features</span><span class="w"> </span><span class="o">--</span><span class="n">features</span><span class="w"> </span><span class="n">rustls</span><span class="o">-</span><span class="n">tls</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">reqwest</span><span class="w"> </span><span class="n">v0</span><span class="mf">.11.4</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span><span class="w"> </span><span class="n">with</span><span class="w"> </span><span class="n">features</span><span class="o">:</span><span class="w"> </span><span class="p">[</span><span class="s">"rustls-tls"</span><span class="p">]</span>
</code></pre></div>
<p>我们准备好发送一个请求了!</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">color_eyre</span>::<span class="n">Report</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing</span>::<span class="n">info</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">EnvFilter</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">reqwest</span>::<span class="n">Client</span><span class="p">;</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello from a comfy nest we've made for ourselves"</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"https://fasterthanli.me"</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// this will turn non-200 HTTP status codes into rust errors,</span>
<span class="w"> </span><span class="c1">// so the first `?` propagates "we had a connection problem" and</span>
<span class="w"> </span><span class="c1">// the second `?` propagates "we had a chat with the server and they</span>
<span class="w"> </span><span class="c1">// were not pleased"</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">setup</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">,</span><span class="w"> </span><span class="s">"1"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">color_eyre</span>::<span class="n">install</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">,</span><span class="w"> </span><span class="s">"info"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">fmt</span>::<span class="n">fmt</span><span class="p">()</span>
<span class="w"> </span><span class="p">.</span><span class="n">with_env_filter</span><span class="p">(</span><span class="n">EnvFilter</span>::<span class="n">from_default_env</span><span class="p">())</span>
<span class="w"> </span><span class="p">.</span><span class="n">init</span><span class="p">();</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>出发了!</p>
<div class="highlight"><pre><span></span><code><span class="nt">cargo</span><span class="w"> </span><span class="nt">run</span>
<span class="w"> </span><span class="nt">Compiling</span><span class="w"> </span><span class="nt">waytoodeep</span><span class="w"> </span><span class="nt">v0</span><span class="p">.</span><span class="nc">1</span><span class="p">.</span><span class="nc">0</span><span class="w"> </span><span class="o">(/</span><span class="nt">Users</span><span class="o">/</span><span class="nt">wh</span><span class="o">/</span><span class="nt">codes</span><span class="o">/</span><span class="nt">rust</span><span class="o">/</span><span class="nt">waytoodeep</span><span class="o">)</span>
<span class="w"> </span><span class="nt">Finished</span><span class="w"> </span><span class="nt">dev</span><span class="w"> </span><span class="cp">[</span><span class="nx">unoptimized</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nx">debuginfo</span><span class="cp">]</span><span class="w"> </span><span class="nt">target</span><span class="o">(</span><span class="nt">s</span><span class="o">)</span><span class="w"> </span><span class="nt">in</span><span class="w"> </span><span class="nt">7</span><span class="p">.</span><span class="nc">16s</span>
<span class="w"> </span><span class="nt">Running</span><span class="w"> </span><span class="err">`</span><span class="nt">target</span><span class="o">/</span><span class="nt">debug</span><span class="o">/</span><span class="nt">waytoodeep</span><span class="err">`</span>
<span class="nt">Jul</span><span class="w"> </span><span class="nt">26</span><span class="w"> </span><span class="nt">16</span><span class="p">:</span><span class="nd">50</span><span class="p">:</span><span class="nd">57</span><span class="p">.</span><span class="nc">778</span><span class="w"> </span><span class="nt">INFO</span><span class="w"> </span><span class="nt">waytoodeep</span><span class="o">:</span><span class="w"> </span><span class="nt">Hello</span><span class="w"> </span><span class="nt">from</span><span class="w"> </span><span class="nt">a</span><span class="w"> </span><span class="nt">comfy</span><span class="w"> </span><span class="nt">nest</span><span class="w"> </span><span class="nt">we</span><span class="err">'</span><span class="nt">ve</span><span class="w"> </span><span class="nt">made</span><span class="w"> </span><span class="nt">for</span><span class="w"> </span><span class="nt">ourselves</span>
<span class="nt">Jul</span><span class="w"> </span><span class="nt">26</span><span class="w"> </span><span class="nt">16</span><span class="p">:</span><span class="nd">50</span><span class="p">:</span><span class="nd">59</span><span class="p">.</span><span class="nc">090</span><span class="w"> </span><span class="nt">INFO</span><span class="w"> </span><span class="nt">waytoodeep</span><span class="o">:</span><span class="w"> </span><span class="nt">Got</span><span class="w"> </span><span class="nt">a</span><span class="w"> </span><span class="nt">response</span><span class="o">!</span><span class="w"> </span><span class="nt">url</span><span class="o">=</span><span class="nt">https</span><span class="o">://</span><span class="nt">fasterthanli</span><span class="p">.</span><span class="nc">me</span><span class="w"> </span><span class="nt">content_type</span><span class="o">=</span><span class="nt">Some</span><span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>这就是我所说的「结构化日志」。嗯,其中的一部分。让我们看下这行代码:</p>
<div class="highlight"><pre><span></span><code><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
</code></pre></div>
<p>我们输出来一个消息: <code>Got a response!</code> ,一个名为 <code>url</code> 的标签:值为变量 <code>url</code> 的 <a href="https://doc.rust-lang.org/stable/std/fmt/trait.Display.html">Display</a> 格式,
一个名为 <code>content_type</code> 的标签:值为表达式的 <a href="https://doc.rust-lang.org/stable/std/fmt/trait.Debug.html">Debug</a> 格式。</p>
<p>就是这么简单! <code>name = %value</code> 输出 <code>Display</code> , <code>name = ?value</code> 输出 <code>Debug</code> 。</p>
<p>当然,还有非常棒的跨度(spans),重点是你可以将它们发送到 APM(Appliation Performance Monitoring),比如 Datadog 或者 Honeycomb 等,但是这不是一篇关于跟踪的文章。</p>
<p>为了举例说明,如果我们安装一个 JSON 的 tracing subscriber,我们将获得如下内容:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">3</span>.09s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
<span class="o">{</span><span class="s2">"timestamp"</span>:<span class="s2">"Jul 25 17:17:21.531"</span>,<span class="s2">"level"</span>:<span class="s2">"INFO"</span>,<span class="s2">"fields"</span>:<span class="o">{</span><span class="s2">"message"</span>:<span class="s2">"Hello from a comfy nest we've made for ourselves"</span><span class="o">}</span>,<span class="s2">"target"</span>:<span class="s2">"waytoodeep"</span><span class="o">}</span>
<span class="o">{</span><span class="s2">"timestamp"</span>:<span class="s2">"Jul 25 17:17:21.709"</span>,<span class="s2">"level"</span>:<span class="s2">"INFO"</span>,<span class="s2">"fields"</span>:<span class="o">{</span><span class="s2">"message"</span>:<span class="s2">"Got a response!"</span>,<span class="s2">"url"</span>:<span class="s2">"https://fasterthanli.me"</span>,<span class="s2">"content_type"</span>:<span class="s2">"Some(\"text/html; charset=utf-8\")"</span><span class="o">}</span>,<span class="s2">"target"</span>:<span class="s2">"waytoodeep"</span><span class="o">}</span>
</code></pre></div>
<p>这应该足以激起你的兴趣。</p>
<h3 id="同时获取两个地址">同时获取两个地址</h3>
<p>现在让我们获取两个地址:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">URL_1</span>: <span class="kp">&</span><span class="kt">str</span> <span class="o">=</span><span class="w"> </span><span class="s">"https://fasterthanli.me/articles/whats-in-the-box"</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">URL_2</span>: <span class="kp">&</span><span class="kt">str</span> <span class="o">=</span><span class="w"> </span><span class="s">"https://fasterthanli.me/series/advent-of-code-2020/part-13"</span><span class="p">;</span>
</code></pre></div>
<p>。。。这是一个公平的比较。 这两篇文章都托管在我自己的网站上,绝对不是为了推广,而是为了使获取时间具有可比性,并且任一都有可能先加载完成(并且会随着时间的推移随机变化)。</p>
<p>我们将创建一个函数来获取内容:</p>
<div class="highlight"><pre><span></span><code><span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">client</span>: <span class="kp">&</span><span class="nc">Client</span><span class="p">,</span><span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>并使用它:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello from a comfy nest we've made for ourselves"</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>然后运行它:</p>
<div class="highlight"><pre><span></span><code><span class="cp">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span>
<span class="w"> </span><span class="n">Compiling</span><span class="w"> </span><span class="n">waytoodeep</span><span class="w"> </span><span class="n">v0</span><span class="p">.</span><span class="mf">1.0</span><span class="w"> </span><span class="p">(</span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="p">)</span>
<span class="n">warning</span>: <span class="nc">unused</span><span class="w"> </span><span class="n">implementer</span><span class="w"> </span><span class="n">of</span><span class="w"> </span><span class="err">`</span><span class="n">Future</span><span class="err">`</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">must</span><span class="w"> </span><span class="n">be</span><span class="w"> </span><span class="n">used</span>
<span class="w"> </span><span class="o">-</span>-> <span class="nc">src</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">rs</span>:<span class="mi">15</span>:<span class="mi">5</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">15</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span>
<span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">note</span>: <span class="err">`</span><span class="cp">#[warn(unused_must_use)]</span><span class="err">`</span><span class="w"> </span><span class="n">on</span><span class="w"> </span><span class="n">by</span><span class="w"> </span><span class="n">default</span>
<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">note</span>: <span class="nc">futures</span><span class="w"> </span><span class="kr">do</span><span class="w"> </span><span class="n">nothing</span><span class="w"> </span><span class="n">unless</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="err">`</span><span class="p">.</span><span class="k">await</span><span class="err">`</span><span class="w"> </span><span class="n">or</span><span class="w"> </span><span class="n">poll</span><span class="w"> </span><span class="n">them</span>
<span class="n">warning</span>: <span class="nc">unused</span><span class="w"> </span><span class="n">implementer</span><span class="w"> </span><span class="n">of</span><span class="w"> </span><span class="err">`</span><span class="n">Future</span><span class="err">`</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">must</span><span class="w"> </span><span class="n">be</span><span class="w"> </span><span class="n">used</span>
<span class="w"> </span><span class="o">-</span>-> <span class="nc">src</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">rs</span>:<span class="mi">16</span>:<span class="mi">5</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">16</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span>
<span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">note</span>: <span class="nc">futures</span><span class="w"> </span><span class="kr">do</span><span class="w"> </span><span class="n">nothing</span><span class="w"> </span><span class="n">unless</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="err">`</span><span class="p">.</span><span class="k">await</span><span class="err">`</span><span class="w"> </span><span class="n">or</span><span class="w"> </span><span class="n">poll</span><span class="w"> </span><span class="n">them</span>
<span class="n">warning</span>: <span class="mi">2</span><span class="w"> </span><span class="n">warnings</span><span class="w"> </span><span class="n">emitted</span>
<span class="w"> </span><span class="n">Finished</span><span class="w"> </span><span class="n">dev</span><span class="w"> </span><span class="p">[</span><span class="n">unoptimized</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">debuginfo</span><span class="p">]</span><span class="w"> </span><span class="n">target</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="mf">3.01</span><span class="n">s</span>
<span class="w"> </span><span class="n">Running</span><span class="w"> </span><span class="err">`</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="err">`</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">17</span>:<span class="mi">26</span>:<span class="mf">31.571</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span>: <span class="nc">Hello</span><span class="w"> </span><span class="n">from</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">comfy</span><span class="w"> </span><span class="n">nest</span><span class="w"> </span><span class="n">we</span><span class="o">'</span><span class="na">ve</span><span class="w"> </span><span class="n">made</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">ourselves</span>
</code></pre></div>
<p>奇怪的是,没有任何事情发生。</p>
<blockquote>
<p>黄色的波浪线和恼人的 Rust 警告已经给出了提示。</p>
</blockquote>
<p>让我们来修复它:</p>
<div class="highlight"><pre><span></span><code><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">3</span>.17s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:27:29.768<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Hello<span class="w"> </span>from<span class="w"> </span>a<span class="w"> </span>comfy<span class="w"> </span>nest<span class="w"> </span>we<span class="err">'</span>ve<span class="w"> </span>made<span class="w"> </span><span class="k">for</span><span class="w"> </span>ourselves
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:27:29.891<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:27:29.974<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>所以,第零课:future 对象不做任何事情直到它们被轮询(polled)。</p>
<p>这是因为 future 对象几乎就是状态。让我们来创建一个:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/main.rs`</span>
<span class="k">mod</span> <span class="nn">dumb</span><span class="p">;</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/dumb.rs`</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="p">{</span>
<span class="w"> </span><span class="n">future</span>::<span class="n">Future</span><span class="p">,</span>
<span class="w"> </span><span class="n">pin</span>::<span class="n">Pin</span><span class="p">,</span>
<span class="w"> </span><span class="n">task</span>::<span class="p">{</span><span class="n">Context</span><span class="p">,</span><span class="w"> </span><span class="n">Poll</span><span class="p">},</span>
<span class="p">};</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing</span>::<span class="n">info</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">DumbFuture</span><span class="w"> </span><span class="p">{}</span>
<span class="k">impl</span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">DumbFuture</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">();</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">_cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello from a dumb future!"</span><span class="p">);</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(())</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// back in `src/main.rs`</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dumb</span>::<span class="n">DumbFuture</span><span class="w"> </span><span class="p">{};</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>以上!我们几乎就完成了,除了我们没有进行 <code>.await</code> 。</p>
<p>运行它除了打印警告不会有任何效果:</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span>
<span class="w"> </span><span class="n">Compiling</span><span class="w"> </span><span class="n">waytoodeep</span><span class="w"> </span><span class="n">v0</span><span class="mf">.1.0</span><span class="w"> </span><span class="p">(</span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="p">)</span>
<span class="n">warning</span><span class="o">:</span><span class="w"> </span><span class="n">unused</span><span class="w"> </span><span class="n">variable</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`fut`</span>
<span class="w"> </span><span class="o">--></span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">14</span><span class="o">:</span><span class="mi">9</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">14</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dumb</span><span class="o">::</span><span class="n">DumbFuture</span><span class="w"> </span><span class="err">{}</span><span class="p">;</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">^^^</span><span class="w"> </span><span class="k">help</span><span class="o">:</span><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">intentional</span><span class="p">,</span><span class="w"> </span><span class="n">prefix</span><span class="w"> </span><span class="n">it</span><span class="w"> </span><span class="k">with</span><span class="w"> </span><span class="n">an</span><span class="w"> </span><span class="n">underscore</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`_fut`</span>
<span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">note</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`#[warn(unused_variables)]`</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="k">default</span>
<span class="n">warning</span><span class="o">:</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">warning</span><span class="w"> </span><span class="n">emitted</span>
<span class="w"> </span><span class="n">Finished</span><span class="w"> </span><span class="n">dev</span><span class="w"> </span><span class="err">[</span><span class="n">unoptimized</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">debuginfo</span><span class="err">]</span><span class="w"> </span><span class="n">target</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="mf">2.11</span><span class="n">s</span>
<span class="w"> </span><span class="n">Running</span><span class="w"> </span><span class="n n-Quoted">`target/debug/waytoodeep`</span>
</code></pre></div>
<p>因为怎么可能?我们字面上仅仅构建了一个结构体。一个零大小的结构体。</p>
<p>如果我们调用它的 <code>.await</code> 。。 然后当我们要求运行时运行它的事件循环直到 future 对象被轮询(polled)并且最终返回 <code>Poll::Ready</code> (我们的代码立即返回):</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Building that dumb future..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dumb</span>::<span class="n">DumbFuture</span><span class="w"> </span><span class="p">{};</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Awaiting that dumb future..."</span><span class="p">);</span>
<span class="w"> </span><span class="n">fut</span><span class="p">.</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Done awaiting that dumb future"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">2</span>.34s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:37:09.261<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:37:09.261<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:37:09.261<span class="w"> </span>INFO<span class="w"> </span>waytoodeep::dumb:<span class="w"> </span>Hello<span class="w"> </span>from<span class="w"> </span>a<span class="w"> </span>dumb<span class="w"> </span>future!
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:37:09.262<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Done<span class="w"> </span>awaiting<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future
</code></pre></div>
<p>这里与 ECMAScript 的 <code>promise</code> 有一些略微的区别:即使它们压根没有被 await 其中包含的工作依然会被执行。</p>
<p>但是 Rust 的 future 对象仅仅是无聊的状态机,如果你故意制造麻烦就可以理解这个机制:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/dumb.rs`</span>
<span class="k">impl</span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">DumbFuture</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">();</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">_cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="fm">panic!</span><span class="p">(</span><span class="s">"Oh heck no"</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">2</span>.28s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:41:18.956<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:41:18.956<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
The<span class="w"> </span>application<span class="w"> </span>panicked<span class="w"> </span><span class="o">(</span>crashed<span class="o">)</span>.
Message:<span class="w"> </span>Oh<span class="w"> </span>heck<span class="w"> </span>no
Location:<span class="w"> </span>src/dumb.rs:14
<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span>BACKTRACE<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
<span class="w"> </span>⋮<span class="w"> </span><span class="m">6</span><span class="w"> </span>frames<span class="w"> </span>hidden<span class="w"> </span>⋮
<span class="w"> </span><span class="m">7</span>:<span class="w"> </span><waytoodeep::dumb::DumbFuture<span class="w"> </span>as<span class="w"> </span>core::future::future::Future>::poll::h4a44780628f4c5f0
<span class="w"> </span>at<span class="w"> </span>/home/amos/ftl/waytoodeep/src/dumb.rs:14
<span class="w"> </span><span class="m">8</span>:<span class="w"> </span>waytoodeep::main::<span class="o">{{</span>closure<span class="o">}}</span>::h36de5a1f1f2a5c5b
<span class="w"> </span>at<span class="w"> </span>/home/amos/ftl/waytoodeep/src/main.rs:17
<span class="w"> </span><span class="m">9</span>:<span class="w"> </span><core::future::from_generator::GenFuture<T><span class="w"> </span>as<span class="w"> </span>core::future::future::Future>::poll::h20a96e082c7a581e
<span class="w"> </span>at<span class="w"> </span>/home/amos/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
<span class="w"> </span><span class="m">10</span>:<span class="w"> </span>tokio::park::thread::CachedParkThread::block_on::<span class="o">{{</span>closure<span class="o">}}</span>::hdf98cb3c7fdf3de4
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263
<span class="w"> </span><span class="m">11</span>:<span class="w"> </span>tokio::coop::with_budget::<span class="o">{{</span>closure<span class="o">}}</span>::h6a86a24a246e220f
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:106
<span class="w"> </span><span class="m">12</span>:<span class="w"> </span>std::thread::local::LocalKey<T>::try_with::h2ce0ac27c85965b6
<span class="w"> </span>at<span class="w"> </span>/home/amos/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:376
<span class="w"> </span><span class="m">13</span>:<span class="w"> </span>std::thread::local::LocalKey<T>::with::hc449f38c9f65fb53
<span class="w"> </span>at<span class="w"> </span>/home/amos/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:352
<span class="w"> </span><span class="m">14</span>:<span class="w"> </span>tokio::coop::with_budget::h5db157bd1e95e0e8
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:99
<span class="w"> </span><span class="m">15</span>:<span class="w"> </span>tokio::coop::budget::h7b57383f1255ac24
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:76
<span class="w"> </span><span class="m">16</span>:<span class="w"> </span>tokio::park::thread::CachedParkThread::block_on::hece399485213b91c
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263
<span class="w"> </span><span class="m">17</span>:<span class="w"> </span>tokio::runtime::enter::Enter::block_on::h89e9882e539e82d3
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/enter.rs:151
<span class="w"> </span><span class="m">18</span>:<span class="w"> </span>tokio::runtime::thread_pool::ThreadPool::block_on::h1a0186470c00ba70
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/thread_pool/mod.rs:71
<span class="w"> </span><span class="m">19</span>:<span class="w"> </span>tokio::runtime::Runtime::block_on::h7c21d6989b86d606
<span class="w"> </span>at<span class="w"> </span>/home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/mod.rs:452
<span class="w"> </span><span class="m">20</span>:<span class="w"> </span>waytoodeep::main::hb4dd5ffd46a5c032
<span class="w"> </span>at<span class="w"> </span>/home/amos/ftl/waytoodeep/src/main.rs:20
<span class="w"> </span><span class="m">21</span>:<span class="w"> </span>core::ops::function::FnOnce::call_once::hc1fcc87431f77d25
<span class="w"> </span>at<span class="w"> </span>/home/amos/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
<span class="w"> </span>⋮<span class="w"> </span><span class="m">11</span><span class="w"> </span>frames<span class="w"> </span>hidden<span class="w"> </span>⋮
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">COLORBT_SHOW_HIDDEN</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>environment<span class="w"> </span>variable<span class="w"> </span>to<span class="w"> </span>disable<span class="w"> </span>frame<span class="w"> </span>filtering.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span>full<span class="w"> </span>to<span class="w"> </span>include<span class="w"> </span><span class="nb">source</span><span class="w"> </span>snippets.
</code></pre></div>
<p>上面堆栈跟踪如果加上颜色效果会更好,所以我希望你在本地做了相同的尝试,即使如此我们依然可以看到我们真正的 main 函数在 20 帧,然后往上,我们可以看到 <code>Runtime::block_on</code> 、一个线程池的东西、一些挂起(parked)的线程、thread-local(其他 TLS)、一个 <strong><em>*生成的</em></strong>* future(帧 9 和 8,也就是我们的 <code>async fn main</code> 的最终结果),最后是我们的 <code>DumbFuture</code> poll 方法(帧 7)。</p>
<p>帧 6 到 1 就是 <a href="https://doc.rust-lang.org/stable/std/panic/index.html">panic</a> 机制,再次完全超出本文讨论的范围。</p>
<p>但是请站起来,亲爱的观众,用你的手臂绕过这个装置,以确保没有障眼法,没有隐藏的线,没有。。。</p>
<p>。。。我要说的是对于异步堆栈跟踪没有“特殊处理”(special handling)。当然,这里我们崩溃了,但是仅仅是 Rust,操作系统甚至不知道我几乎避免了一场灾难。</p>
<p>但是我们可以制造更大的混乱,如果我们愿意使用 <code>unsafe</code> :</p>
<div class="highlight"><pre><span></span><code><span class="k">impl</span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">DumbFuture</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">();</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">_cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="o">*</span><span class="p">(</span><span class="mh">0xF00D</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="kt">u64</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0x0</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">unreachable!</span><span class="p">();</span><span class="w"> </span><span class="c1">// pinky promise</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>然后就不会有一些列的崩溃处理来拯救我们:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">2</span>.18s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:46:53.926<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:46:53.926<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>dumb<span class="w"> </span>future...
zsh:<span class="w"> </span>segmentation<span class="w"> </span>fault<span class="w"> </span><span class="o">(</span>core<span class="w"> </span>dumped<span class="o">)</span><span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>cargo<span class="w"> </span>run
</code></pre></div>
<p>但是 GDB 可以:</p>
<div class="highlight"><pre><span></span><code><span class="o">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">build</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">gdb</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">--</span><span class="n">args</span><span class="w"> </span><span class="o">./</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span>
<span class="w"> </span><span class="n">Finished</span><span class="w"> </span><span class="n">dev</span><span class="w"> </span><span class="p">[</span><span class="n">unoptimized</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">debuginfo</span><span class="p">]</span><span class="w"> </span><span class="n">target</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="mf">0.04</span><span class="n">s</span>
<span class="n">Reading</span><span class="w"> </span><span class="n">symbols</span><span class="w"> </span><span class="n">from</span><span class="w"> </span><span class="o">./</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="o">...</span>
<span class="n">warning</span><span class="p">:</span><span class="w"> </span><span class="n">Missing</span><span class="w"> </span><span class="n">auto</span><span class="o">-</span><span class="nb">load</span><span class="w"> </span><span class="n">script</span><span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="n">offset</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">section</span><span class="w"> </span><span class="o">.</span><span class="n">debug_gdb_scripts</span>
<span class="n">of</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="o">/</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="o">.</span>
<span class="n">Use</span><span class="w"> </span><span class="err">`</span><span class="n">info</span><span class="w"> </span><span class="n">auto</span><span class="o">-</span><span class="nb">load</span><span class="w"> </span><span class="n">python</span><span class="o">-</span><span class="n">scripts</span><span class="w"> </span><span class="p">[</span><span class="n">REGEXP</span><span class="p">]</span><span class="s1">' to list them.</span>
<span class="p">(</span><span class="n">gdb</span><span class="p">)</span><span class="w"> </span><span class="n">r</span>
<span class="n">Starting</span><span class="w"> </span><span class="n">program</span><span class="p">:</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="o">/</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span>
<span class="p">[</span><span class="n">Thread</span><span class="w"> </span><span class="n">debugging</span><span class="w"> </span><span class="n">using</span><span class="w"> </span><span class="n">libthread_db</span><span class="w"> </span><span class="n">enabled</span><span class="p">]</span>
<span class="n">Using</span><span class="w"> </span><span class="n">host</span><span class="w"> </span><span class="n">libthread_db</span><span class="w"> </span><span class="n">library</span><span class="w"> </span><span class="s2">"/lib/x86_64-linux-gnu/libthread_db.so.1"</span><span class="o">.</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7c28700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129418</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7a27700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129419</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7826700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129420</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7625700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129421</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7424700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129422</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7223700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129423</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7022700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129424</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6e1e700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129425</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6c1a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129426</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6a16700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129427</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6812700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129428</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff660e700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129429</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff640a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129430</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6206700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129431</span><span class="p">)]</span>
<span class="p">[</span><span class="n">New</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6002700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">129432</span><span class="p">)]</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">17</span><span class="p">:</span><span class="mi">47</span><span class="p">:</span><span class="mf">13.278</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Building</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">dumb</span><span class="w"> </span><span class="n">future</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">17</span><span class="p">:</span><span class="mi">47</span><span class="p">:</span><span class="mf">13.279</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Awaiting</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">dumb</span><span class="w"> </span><span class="n">future</span><span class="o">...</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="s2">"waytoodeep"</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="k">signal</span><span class="w"> </span><span class="n">SIGSEGV</span><span class="p">,</span><span class="w"> </span><span class="n">Segmentation</span><span class="w"> </span><span class="n">fault</span><span class="o">.</span>
<span class="o"><</span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">dumb</span><span class="p">::</span><span class="n">DumbFuture</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">core</span><span class="p">::</span><span class="n">future</span><span class="p">::</span><span class="n">future</span><span class="p">::</span><span class="n">Future</span><span class="o">></span><span class="p">::</span><span class="n">poll</span><span class="w"> </span><span class="p">(</span><span class="bp">self</span><span class="o">=...</span><span class="p">,</span><span class="w"> </span><span class="n">_cx</span><span class="o">=</span><span class="mh">0x7fffffffd690</span><span class="p">)</span><span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">dumb</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">15</span>
<span class="mi">15</span><span class="w"> </span><span class="o">*</span><span class="p">(</span><span class="mh">0xF00D</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="o">*</span><span class="n">mut</span><span class="w"> </span><span class="n">u64</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0x0</span><span class="p">;</span>
<span class="p">(</span><span class="n">gdb</span><span class="p">)</span><span class="w"> </span><span class="n">bt</span>
<span class="c1">#0 <waytoodeep::dumb::DumbFuture as core::future::future::Future>::poll (self=..., _cx=0x7fffffffd690) at src/dumb.rs:15</span>
<span class="c1">#1 0x00005555555ab3a3 in waytoodeep::main::{{closure}} () at src/main.rs:17</span>
<span class="c1">#2 0x00005555555adb29 in <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (self=..., cx=0x7fffffffd690)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">rustup</span><span class="o">/</span><span class="n">toolchains</span><span class="o">/</span><span class="n">stable</span><span class="o">-</span><span class="n">x86_64</span><span class="o">-</span><span class="n">unknown</span><span class="o">-</span><span class="n">linux</span><span class="o">-</span><span class="n">gnu</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">rustlib</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">rust</span><span class="o">/</span><span class="n">library</span><span class="o">/</span><span class="n">core</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">future</span><span class="o">/</span><span class="n">mod</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">80</span>
<span class="c1">#3 0x00005555555adaa0 in tokio::park::thread::CachedParkThread::block_on::{{closure}} ()</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">park</span><span class="o">/</span><span class="n">thread</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">263</span>
<span class="c1">#4 0x00005555555b1742 in tokio::coop::with_budget::{{closure}} (cell=0x7ffff7c2c412)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">coop</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">106</span>
<span class="c1">#5 0x00005555555a9f58 in std::thread::local::LocalKey<T>::try_with (self=0x555555925fc0, f=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">rustup</span><span class="o">/</span><span class="n">toolchains</span><span class="o">/</span><span class="n">stable</span><span class="o">-</span><span class="n">x86_64</span><span class="o">-</span><span class="n">unknown</span><span class="o">-</span><span class="n">linux</span><span class="o">-</span><span class="n">gnu</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">rustlib</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">rust</span><span class="o">/</span><span class="n">library</span><span class="o">/</span><span class="n">std</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread</span><span class="o">/</span><span class="n">local</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">376</span>
<span class="c1">#6 0x00005555555a9e3d in std::thread::local::LocalKey<T>::with (self=0x555555925fc0, f=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">rustup</span><span class="o">/</span><span class="n">toolchains</span><span class="o">/</span><span class="n">stable</span><span class="o">-</span><span class="n">x86_64</span><span class="o">-</span><span class="n">unknown</span><span class="o">-</span><span class="n">linux</span><span class="o">-</span><span class="n">gnu</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">rustlib</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">rust</span><span class="o">/</span><span class="n">library</span><span class="o">/</span><span class="n">std</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread</span><span class="o">/</span><span class="n">local</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">352</span>
<span class="c1">#7 0x00005555555ad7c8 in tokio::coop::with_budget (budget=..., f=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">coop</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">99</span>
<span class="c1">#8 tokio::coop::budget (f=...) at /home/amos/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:76</span>
<span class="c1">#9 tokio::park::thread::CachedParkThread::block_on (self=0x7fffffffd7a0, f=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">park</span><span class="o">/</span><span class="n">thread</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">263</span>
<span class="c1">#10 0x00005555555abcc9 in tokio::runtime::enter::Enter::block_on (self=0x7fffffffd7f0, f=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">runtime</span><span class="o">/</span><span class="n">enter</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">151</span>
<span class="c1">#11 0x00005555555acf2e in tokio::runtime::thread_pool::ThreadPool::block_on (self=0x7fffffffd908, future=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">runtime</span><span class="o">/</span><span class="n">thread_pool</span><span class="o">/</span><span class="n">mod</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">71</span>
<span class="c1">#12 0x00005555555b0dfd in tokio::runtime::Runtime::block_on (self=0x7fffffffd900, future=...)</span>
<span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">tokio</span><span class="o">-</span><span class="mf">1.9</span><span class="o">.</span><span class="mi">0</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">runtime</span><span class="o">/</span><span class="n">mod</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">452</span>
<span class="c1">#13 0x00005555555aa807 in waytoodeep::main () at src/main.rs:20</span>
<span class="p">(</span><span class="n">gdb</span><span class="p">)</span>
</code></pre></div>
<p>我们再次丢失了高亮颜色,这里可以看一下:
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/gdb-colors.b45af429c46a37d9.webp"></p>
<blockquote>
<p>译注:我在本地环境并没有通过 GDB 复现带高亮的堆栈跟踪,反而是通过 LLDB 可以看到高亮的堆栈跟踪。</p>
</blockquote>
<p>是不是很漂亮?</p>
<p>现在让我们回到正常有用的代码,移除所有关于自己实现的 future 代码: <code>src/dumb.rs</code> 和 <code>mod dumb</code> 。并使用一个获取 future 替代:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Building that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Awaiting that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="n">fut</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Done awaiting that fetch future"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">2</span>.99s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:51:49.281<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:51:49.282<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:51:49.437<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">17</span>:51:49.438<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Done<span class="w"> </span>awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future
</code></pre></div>
<p>有两种方式考虑我们的函数,一个是语法糖层:也就是 <code>async fn</code> :</p>
<div class="highlight"><pre><span></span><code><span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">client</span>: <span class="kp">&</span><span class="nc">Client</span><span class="p">,</span><span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>然后是核心实现层:一个普通的 <code>fn</code> 仅用来返回一个 future 对象:</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">future</span>::<span class="n">Future</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">fetch_thing</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span>
<span class="w"> </span><span class="n">client</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="na">a</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>由于借用 <code>client</code> 和 <code>url</code> ,所以 <code>Future</code> 对象的存活时间不能超过两者,这也是为什么我会将上面两个生命周期命名为 <code>'a</code> ,
并且返回的值也是任意实现了 <code>Future</code> (通过 <code>Output</code> )同时生命周期也是 <code>'a</code> 。</p>
<p>整个 <code>async move {}</code> 快也仅仅是“构建状态” -- 等于一个实现了 <code>Future</code> 的类型。</p>
<p>我们只是无法命名它。</p>
<p>我们只能尽量获取它的描述:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">type_name_of</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="n">_</span>: <span class="kp">&</span><span class="nc">T</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="kt">str</span> <span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">any</span>::<span class="n">type_name</span>::<span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">()</span>
<span class="p">}</span>
<span class="c1">// in main</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Building that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span>
<span class="w"> </span><span class="n">type_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">type_name_of</span><span class="p">(</span><span class="o">&</span><span class="n">fut</span><span class="p">),</span>
<span class="w"> </span><span class="s">"That fetch future has a type.."</span>
<span class="w"> </span><span class="p">);</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Awaiting that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="n">fut</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Done awaiting that fetch future"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.05s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:00:39.774<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:00:39.775<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>That<span class="w"> </span>fetch<span class="w"> </span>future<span class="w"> </span>has<span class="w"> </span>a<span class="w"> </span>type..<span class="w"> </span><span class="nv">type_name</span><span class="o">=</span><span class="s2">"core::future::from_generator::GenFuture<waytoodeep::fetch_thing::{{closure}}>"</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:00:39.775<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:00:39.882<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:00:39.882<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Done<span class="w"> </span>awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future
</code></pre></div>
<p>。。。但是等等,由于我们使用了 <code>async</code> 语法所以它是一个编译器生成的类型。某种意义上我们无法命名它也就意味这我们无法绑定这个对象,或者编写一个函数仅仅接受该类型。</p>
<p>为了让我们自己相信 future 对象在我们真正轮询它之前它不会做任何工作,我们可以打开 <code>reqwest</code> 的调试日志:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info,reqwest<span class="o">=</span>debug<span class="w"> </span>cargo<span class="w"> </span>run
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">3</span>.07s
<span class="w"> </span>Running<span class="w"> </span><span class="sb">`</span>target/debug/waytoodeep<span class="sb">`</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.384<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.385<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>That<span class="w"> </span>fetch<span class="w"> </span>future<span class="w"> </span>has<span class="w"> </span>a<span class="w"> </span>type..<span class="w"> </span><span class="nv">type_name</span><span class="o">=</span><span class="s2">"core::future::from_generator::GenFuture<waytoodeep::fetch_thing::{{closure}}>"</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.385<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.385<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::connect:<span class="w"> </span>starting<span class="w"> </span>new<span class="w"> </span>connection:<span class="w"> </span>https://fasterthanli.me/
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.503<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/articles/whats-in-the-box
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.503<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:05:07.503<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Done<span class="w"> </span>awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future
</code></pre></div>
<p>甚至对于每一个包(crate),我们都可以通过监听 <a href="https://lib.rs/crates/hyper">hyper</a> 和 <a href="https://lib.rs/crates/h2">h2</a> 来观察:</p>
<div class="highlight"><pre><span></span><code><span class="o">$</span><span class="w"> </span><span class="n">RUST_LOG</span><span class="o">=</span><span class="n">debug</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span>
<span class="w"> </span><span class="n">Finished</span><span class="w"> </span><span class="n">dev</span><span class="w"> </span><span class="p">[</span><span class="n">unoptimized</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">debuginfo</span><span class="p">]</span><span class="w"> </span><span class="n">target</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="mf">0.04</span><span class="n">s</span>
<span class="w"> </span><span class="n">Running</span><span class="w"> </span><span class="err">`</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="err">`</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.973</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Building</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">fetch</span><span class="w"> </span><span class="n">future</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.973</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">That</span><span class="w"> </span><span class="n">fetch</span><span class="w"> </span><span class="n">future</span><span class="w"> </span><span class="n">has</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">type</span><span class="o">..</span><span class="w"> </span><span class="n">type_name</span><span class="o">=</span><span class="s2">"core::future::from_generator::GenFuture<waytoodeep::fetch_thing::{{closure}}>"</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.973</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Awaiting</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">fetch</span><span class="w"> </span><span class="n">future</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.974</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">reqwest</span><span class="p">::</span><span class="n">connect</span><span class="p">:</span><span class="w"> </span><span class="n">starting</span><span class="w"> </span><span class="n">new</span><span class="w"> </span><span class="n">connection</span><span class="p">:</span><span class="w"> </span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">fasterthanli</span><span class="o">.</span><span class="n">me</span><span class="o">/</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.974</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">hyper</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">connect</span><span class="p">::</span><span class="n">dns</span><span class="p">:</span><span class="w"> </span><span class="n">resolving</span><span class="w"> </span><span class="n">host</span><span class="o">=</span><span class="s2">"fasterthanli.me"</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mf">59.989</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">hyper</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">connect</span><span class="p">::</span><span class="n">http</span><span class="p">:</span><span class="w"> </span><span class="n">connecting</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="mf">172.67</span><span class="o">.</span><span class="mf">196.144</span><span class="p">:</span><span class="mi">443</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.000</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">hyper</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">connect</span><span class="p">::</span><span class="n">http</span><span class="p">:</span><span class="w"> </span><span class="n">connected</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="mf">172.67</span><span class="o">.</span><span class="mf">196.144</span><span class="p">:</span><span class="mi">443</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.000</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">hs</span><span class="p">:</span><span class="w"> </span><span class="n">No</span><span class="w"> </span><span class="n">cached</span><span class="w"> </span><span class="n">session</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">DNSNameRef</span><span class="p">(</span><span class="s2">"fasterthanli.me"</span><span class="p">)</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.000</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">hs</span><span class="p">:</span><span class="w"> </span><span class="n">Not</span><span class="w"> </span><span class="n">resuming</span><span class="w"> </span><span class="n">any</span><span class="w"> </span><span class="n">session</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.016</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">hs</span><span class="p">:</span><span class="w"> </span><span class="n">Using</span><span class="w"> </span><span class="n">ciphersuite</span><span class="w"> </span><span class="n">TLS13_CHACHA20_POLY1305_SHA256</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.016</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">tls13</span><span class="p">:</span><span class="w"> </span><span class="n">Not</span><span class="w"> </span><span class="n">resuming</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.017</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">tls13</span><span class="p">:</span><span class="w"> </span><span class="n">TLS1</span><span class="o">.</span><span class="mi">3</span><span class="w"> </span><span class="n">encrypted</span><span class="w"> </span><span class="n">extensions</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="n">ServerNameAck</span><span class="p">,</span><span class="w"> </span><span class="n">Protocols</span><span class="p">([</span><span class="n">PayloadU8</span><span class="p">([</span><span class="mi">104</span><span class="p">,</span><span class="w"> </span><span class="mi">50</span><span class="p">])])]</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.017</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">hs</span><span class="p">:</span><span class="w"> </span><span class="n">ALPN</span><span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">Some</span><span class="p">(</span><span class="sa">b</span><span class="s2">"h2"</span><span class="p">)</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.018</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">client</span><span class="p">:</span><span class="w"> </span><span class="n">binding</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="n">connection</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.018</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">client</span><span class="p">:</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="n">connection</span><span class="w"> </span><span class="n">bound</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.018</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Settings</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x0</span><span class="p">),</span><span class="w"> </span><span class="n">enable_push</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">initial_window_size</span><span class="p">:</span><span class="w"> </span><span class="mi">2097152</span><span class="p">,</span><span class="w"> </span><span class="n">max_frame_size</span><span class="p">:</span><span class="w"> </span><span class="mi">16384</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.019</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">WindowUpdate</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span><span class="w"> </span><span class="n">size_increment</span><span class="p">:</span><span class="w"> </span><span class="mi">5177345</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.019</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">hyper</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">pool</span><span class="p">:</span><span class="w"> </span><span class="n">pooling</span><span class="w"> </span><span class="n">idle</span><span class="w"> </span><span class="n">connection</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="s2">"https"</span><span class="p">,</span><span class="w"> </span><span class="n">fasterthanli</span><span class="o">.</span><span class="n">me</span><span class="p">)</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.020</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Headers</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x5</span><span class="p">:</span><span class="w"> </span><span class="n">END_HEADERS</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">END_STREAM</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.029</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">tls13</span><span class="p">:</span><span class="w"> </span><span class="n">Ticket</span><span class="w"> </span><span class="n">saved</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.029</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">client</span><span class="p">::</span><span class="n">tls13</span><span class="p">:</span><span class="w"> </span><span class="n">Ticket</span><span class="w"> </span><span class="n">saved</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.029</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Settings</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x0</span><span class="p">),</span><span class="w"> </span><span class="n">max_concurrent_streams</span><span class="p">:</span><span class="w"> </span><span class="mi">256</span><span class="p">,</span><span class="w"> </span><span class="n">initial_window_size</span><span class="p">:</span><span class="w"> </span><span class="mi">65536</span><span class="p">,</span><span class="w"> </span><span class="n">max_frame_size</span><span class="p">:</span><span class="w"> </span><span class="mi">16777215</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.030</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Settings</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x1</span><span class="p">:</span><span class="w"> </span><span class="n">ACK</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.030</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">WindowUpdate</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span><span class="w"> </span><span class="n">size_increment</span><span class="p">:</span><span class="w"> </span><span class="mi">2147418112</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.041</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Settings</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x1</span><span class="p">:</span><span class="w"> </span><span class="n">ACK</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.041</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">proto</span><span class="p">::</span><span class="n">settings</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">settings</span><span class="w"> </span><span class="n">ACK</span><span class="p">;</span><span class="w"> </span><span class="n">applying</span><span class="w"> </span><span class="n">Settings</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x0</span><span class="p">),</span><span class="w"> </span><span class="n">enable_push</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">initial_window_size</span><span class="p">:</span><span class="w"> </span><span class="mi">2097152</span><span class="p">,</span><span class="w"> </span><span class="n">max_frame_size</span><span class="p">:</span><span class="w"> </span><span class="mi">16384</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.120</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Headers</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span><span class="w"> </span><span class="n">flags</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="mh">0x4</span><span class="p">:</span><span class="w"> </span><span class="n">END_HEADERS</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.120</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Data</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.121</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">reqwest</span><span class="p">::</span><span class="n">async_impl</span><span class="p">::</span><span class="n">client</span><span class="p">:</span><span class="w"> </span><span class="n">response</span><span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">fasterthanli</span><span class="o">.</span><span class="n">me</span><span class="o">/</span><span class="n">articles</span><span class="o">/</span><span class="n">whats</span><span class="o">-</span><span class="ow">in</span><span class="o">-</span><span class="n">the</span><span class="o">-</span><span class="n">box</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.121</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Got</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">response</span><span class="o">!</span><span class="w"> </span><span class="n">url</span><span class="o">=</span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">fasterthanli</span><span class="o">.</span><span class="n">me</span><span class="o">/</span><span class="n">articles</span><span class="o">/</span><span class="n">whats</span><span class="o">-</span><span class="ow">in</span><span class="o">-</span><span class="n">the</span><span class="o">-</span><span class="n">box</span><span class="w"> </span><span class="n">content_type</span><span class="o">=</span><span class="n">Some</span><span class="p">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="p">)</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.121</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Done</span><span class="w"> </span><span class="n">awaiting</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">fetch</span><span class="w"> </span><span class="n">future</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.121</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_read</span><span class="p">:</span><span class="w"> </span><span class="n">received</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Data</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.122</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">Reset</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span><span class="w"> </span><span class="n">error_code</span><span class="p">:</span><span class="w"> </span><span class="n">CANCEL</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.122</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">codec</span><span class="p">::</span><span class="n">framed_write</span><span class="p">:</span><span class="w"> </span><span class="n">send</span><span class="w"> </span><span class="n">frame</span><span class="o">=</span><span class="n">GoAway</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">error_code</span><span class="p">:</span><span class="w"> </span><span class="n">NO_ERROR</span><span class="p">,</span><span class="w"> </span><span class="n">last_stream_id</span><span class="p">:</span><span class="w"> </span><span class="n">StreamId</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="w"> </span><span class="p">}</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.122</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">h2</span><span class="p">::</span><span class="n">proto</span><span class="p">::</span><span class="n">connection</span><span class="p">:</span><span class="w"> </span><span class="n">Connection</span><span class="p">::</span><span class="n">poll</span><span class="p">;</span><span class="w"> </span><span class="n">connection</span><span class="w"> </span><span class="n">error</span><span class="w"> </span><span class="n">error</span><span class="o">=</span><span class="n">NO_ERROR</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">18</span><span class="p">:</span><span class="mi">06</span><span class="p">:</span><span class="mf">00.122</span><span class="w"> </span><span class="n">DEBUG</span><span class="w"> </span><span class="n">Connection</span><span class="p">{</span><span class="n">peer</span><span class="o">=</span><span class="n">Client</span><span class="p">}:</span><span class="w"> </span><span class="n">rustls</span><span class="p">::</span><span class="n">session</span><span class="p">:</span><span class="w"> </span><span class="n">Sending</span><span class="w"> </span><span class="n">warning</span><span class="w"> </span><span class="n">alert</span><span class="w"> </span><span class="n">CloseNotify</span>
</code></pre></div>
<blockquote>
<p>上面出现了 rustls,并且使用了 TLS 1.3,作者做过<a href="https://www.youtube.com/watch?v=YHIiVsFybLA">一期视频</a>介绍过 TLS 1.3。</p>
</blockquote>
<p>这些应该足够说服你,除非你只相信内核所说的,所以让我们看看调用堆栈只为了更加确定。</p>
<p>我们在 <code>await</code> future 对象之前增加一秒钟的休眠:</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">time</span>::<span class="n">sleep</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">time</span>::<span class="n">Duration</span><span class="p">;</span>
<span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Building that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Sleeping for a bit..."</span><span class="p">);</span>
<span class="w"> </span><span class="n">sleep</span><span class="p">(</span><span class="n">Duration</span>::<span class="n">from_secs</span><span class="p">(</span><span class="mi">1</span><span class="p">)).</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Awaiting that fetch future..."</span><span class="p">);</span>
<span class="w"> </span><span class="n">fut</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Done awaiting that fetch future"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>build<span class="w"> </span><span class="o">&&</span><span class="w"> </span>strace<span class="w"> </span>-e<span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>./target/debug/waytoodeep
<span class="w"> </span>Compiling<span class="w"> </span>waytoodeep<span class="w"> </span>v0.1.0<span class="w"> </span><span class="o">(</span>/home/amos/ftl/waytoodeep<span class="o">)</span>
<span class="w"> </span>Finished<span class="w"> </span>dev<span class="w"> </span><span class="o">[</span>unoptimized<span class="w"> </span>+<span class="w"> </span>debuginfo<span class="o">]</span><span class="w"> </span>target<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">3</span>.13s
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:09:36.595<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Building<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:09:36.596<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Sleeping<span class="w"> </span><span class="k">for</span><span class="w"> </span>a<span class="w"> </span>bit...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:09:37.599<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future...
connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"104.21.92.169"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:09:37.720<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:09:37.721<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Done<span class="w"> </span>awaiting<span class="w"> </span>that<span class="w"> </span>fetch<span class="w"> </span>future
+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
</code></pre></div>
<p>再次强调,附上会让显著提高上面信息的可读性,如果不让我选择它们的话我是非常喜欢高亮的。我本地看起来是这样的:
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/strace-colors.a4163f4bda179c2b.webp">
由于 <code>tracing-subscriber</code> 默认格式会输出时间戳,可以看到程序休眠了1分钟(外加3毫秒),而且只有我们真正调用 <code>await</code> 时我们的程序才会开始连接到托管文章的 CDN 节点。</p>
<p>好了!让我们再次尝试拉取两篇文章:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="n">fut1</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">fut2</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>再次检查日志:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info,reqwest<span class="o">=</span>debug<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:31:47.396<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::connect:<span class="w"> </span>starting<span class="w"> </span>new<span class="w"> </span>connection:<span class="w"> </span>https://fasterthanli.me/
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:31:47.536<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/articles/whats-in-the-box
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:31:47.537<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:31:47.627<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/series/advent-of-code-2020/part-13
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:31:47.627<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>非常有趣。从这里可以看到, <code>reqwest</code> 为两个请求复用了相同的连接。我会这么说是因我只看到了一行 <code>reqwest::connect</code> 日志。</p>
<p>让我们快速通过 <code>strace</code> 检查一下:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>build<span class="w"> </span>--quiet<span class="w"> </span><span class="o">&&</span><span class="w"> </span>strace<span class="w"> </span>-e<span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>./target/debug/waytoodeep<span class="w"> </span>><span class="w"> </span>/dev/null
connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"172.67.196.144"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
</code></pre></div>
<p>现在可以确认了,只有一次连接。</p>
<p>但是,第一个请求完成后才开始了第二个请求。第一个耗费了 <code>536-396 = 140</code> 毫秒,但是第二个耗费了 <code>627-537 = 90</code> 毫秒!</p>
<blockquote>
<p>Emmm,现在我们运行构建的是 debug 版本不是吗?</p>
</blockquote>
<p>这是真的。我确信我们面临的是 IO 密集型,而不是 CPU 密集型。</p>
<p>debug 版本的构建绝对有一些额外的开销,但是我怀疑这里它不会太影响延迟。无论如何,让我们检查一下:
(注意 --release)</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info,reqwest<span class="o">=</span>debug<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:34:59.211<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::connect:<span class="w"> </span>starting<span class="w"> </span>new<span class="w"> </span>connection:<span class="w"> </span>https://fasterthanli.me/
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:34:59.343<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/articles/whats-in-the-box
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:34:59.343<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:34:59.427<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/series/advent-of-code-2020/part-13
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:34:59.427<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>我们计算一下延迟 <code>343-211 = 132ms</code> , <code>427-343 = 84ms</code> 。</p>
<p>几毫秒的差异可能的解释是邻居打开了一个 YouTube 视频导致无线电波爆发,从而导致冲突(802.11 没有空中流量控制,全民自由(free-for-all))和重传。</p>
<p>或者另外一百万个原因。这也是我们不继续分析的原因。</p>
<p>让我们回到文章的主题。</p>
<h3 id="等待第一个完成">等待第一个完成</h3>
<p>是的!等待第一个完成。所以我们如何让程序同时请求两个?</p>
<p>其实有一大堆方式!</p>
<p>例如,我们可以在一个执行器上执行( <code>spawn</code> )这些 future 对象,然后休眠一秒钟。1 秒钟足够了吧?</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">time</span>::<span class="n">sleep</span><span class="p">(</span><span class="n">Duration</span>::<span class="n">from_secs</span><span class="p">(</span><span class="mi">1</span><span class="p">)).</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">RUST_LOG</span><span class="o">=</span><span class="n">info</span><span class="p">,</span><span class="n">reqwest</span><span class="o">=</span><span class="n">debug</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">--</span><span class="k">release</span>
<span class="k">error</span><span class="err">[</span><span class="n">E0597</span><span class="err">]</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`client`</span><span class="w"> </span><span class="n">does</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="n">live</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">enough</span>
<span class="w"> </span><span class="o">--></span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">17</span><span class="o">:</span><span class="mi">28</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">17</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="o">&</span><span class="k">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">------------^^^^^^^--------</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">borrowed</span><span class="w"> </span><span class="k">value</span><span class="w"> </span><span class="n">does</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="n">live</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">enough</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">argument</span><span class="w"> </span><span class="n">requires</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n n-Quoted">`client`</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">borrowed</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n n-Quoted">`'static`</span>
<span class="p">...</span>
<span class="mi">25</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n n-Quoted">`client`</span><span class="w"> </span><span class="n">dropped</span><span class="w"> </span><span class="n">here</span><span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="n">still</span><span class="w"> </span><span class="n">borrowed</span>
<span class="k">error</span><span class="o">:</span><span class="w"> </span><span class="n">aborting</span><span class="w"> </span><span class="n">due</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">previous</span><span class="w"> </span><span class="k">error</span>
<span class="k">For</span><span class="w"> </span><span class="n">more</span><span class="w"> </span><span class="n">information</span><span class="w"> </span><span class="n">about</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="k">error</span><span class="p">,</span><span class="w"> </span><span class="n">try</span><span class="w"> </span><span class="n n-Quoted">`rustc --explain E0597`</span><span class="p">.</span>
<span class="k">error</span><span class="o">:</span><span class="w"> </span><span class="n">could</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="n">compile</span><span class="w"> </span><span class="n n-Quoted">`waytoodeep`</span>
<span class="k">To</span><span class="w"> </span><span class="n">learn</span><span class="w"> </span><span class="n">more</span><span class="p">,</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">command</span><span class="w"> </span><span class="n">again</span><span class="w"> </span><span class="k">with</span><span class="w"> </span><span class="o">--</span><span class="n">verbose</span><span class="p">.</span>
</code></pre></div>
<p>额,除非我们不可以。不可以是因为。。。</p>
<blockquote>
<p>我们将「future 对象交给执行器执行」并将 future 对象转交给执行器,对吧?我们转移了它和它的内容的所有权。</p>
<p>然后即使我们不对其进行 <code>await</code> ,future 对象因为是「执行器需要做」的一部分依然会被执行,所以即使我们从 <code>main</code> 返回 future 对象也会被轮询(polled)。</p>
<p>但是如果我们从 <code>main</code> 返回,则整个程序都会退出。</p>
<p>这里也可以是任何函数(这里是 <code>main</code> )。重要的是如果函数返回了但是 future 对象借用了部分数据将无法通过借用检查器。</p>
</blockquote>
<p>这让我很高兴,因为这意味着我们不会意外访问到一些被释放的资源:<a href="https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=use+after+free">UAF</a>。</p>
<p>这里我们的例子没有完成。</p>
<p>所以。。。我们需要解决这个问题。如果 <code>fetch_thing</code> 返回的 future 对象是 <code>'static</code> 的呢?或者它不借用任何东西?</p>
<p>程序现在看起来如下:</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">future</span>::<span class="n">Future</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">fetch_thing</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span>
<span class="w"> </span><span class="n">client</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="na">a</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>好吧,之前我们用了 <code>async fn</code> ,但是为了更加深入的理解,我们不得不放弃漂亮的语法。</p>
<p>但是幸运的是,这正是我们想要的:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span>
<span class="w"> </span><span class="n">client</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="kt">str</span><span class="p">,</span>
<span class="c1">// 👇</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="nb">static</span><span class="w"> </span><span class="p">{}</span>
</code></pre></div>
<p>但是我们借用了 <code>client</code> 和 <code>url</code> 我们必须避免这个问题。</p>
<p>因为 <code>url</code> 本身就是常量,所以很容易解决:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">URL_1</span>: <span class="kp">&</span><span class="kt">str</span> <span class="o">=</span><span class="w"> </span><span class="s">"https://fasterthanli.me/articles/whats-in-the-box"</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">URL_2</span>: <span class="kp">&</span><span class="kt">str</span> <span class="o">=</span><span class="w"> </span><span class="s">"https://fasterthanli.me/series/advent-of-code-2020/part-13"</span><span class="p">;</span>
</code></pre></div>
<p>它们本身就是 <code>'static</code> 。所以我们只需要调整需要 <code>'static</code> 就行:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span>
<span class="w"> </span><span class="n">client</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="nb">static</span><span class="w"> </span><span class="p">{}</span>
</code></pre></div>
<p>非常好!解决了一个生命周期,还剩下一个。</p>
<p>我们可以要求 <code>client</code> 的生命周期为 <code>'static</code> 。由于它是一个 <code>Client</code> 的引用,意味着 <code>Cleint</code> 本身也需要是 <code>'static</code> 生命周期。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">client</span>: <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="nb">static</span><span class="w"> </span><span class="p">{}</span>
</code></pre></div>
<p>由于它被 <code>main</code> 所有,额,我们可以,可以。。。可以泄漏它:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">leaked_client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Box</span>::<span class="n">leak</span><span class="p">(</span><span class="nb">Box</span>::<span class="n">new</span><span class="p">(</span><span class="n">client</span><span class="p">));</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">leaked_client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">leaked_client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">tokio</span>::<span class="n">time</span>::<span class="n">sleep</span><span class="p">(</span><span class="n">Duration</span>::<span class="n">from_secs</span><span class="p">(</span><span class="mi">1</span><span class="p">)).</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>完美!没有生命周期的问题了。</p>
<p>仅仅将所有东西泄漏就行。看到没?你不需要 C!</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info,reqwest<span class="o">=</span>debug<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.614<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::connect:<span class="w"> </span>starting<span class="w"> </span>new<span class="w"> </span>connection:<span class="w"> </span>https://fasterthanli.me/
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.614<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::connect:<span class="w"> </span>starting<span class="w"> </span>new<span class="w"> </span>connection:<span class="w"> </span>https://fasterthanli.me/
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.708<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/articles/whats-in-the-box
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.708<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.733<span class="w"> </span>DEBUG<span class="w"> </span>reqwest::async_impl::client:<span class="w"> </span>response<span class="w"> </span><span class="s1">'200 OK'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>https://fasterthanli.me/series/advent-of-code-2020/part-13
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:54:53.733<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>非~~常有趣!</p>
<p>我们的两个请求肯定是并发的发出去了,我们之所以知道是因为从我的笔记本上请求我的网站大概耗时 80ms 到 140ms 之间,但是在日志中我们看到两个响应之间只有 ~25ms 的间隔。</p>
<p>我们还可以看到 <code>reqwest</code> 有连接池机制:同时创建了两个连接。可能是因为我们开始第二个连接的时候第一个请求的连接还没有建立完成。</p>
<p>也就意味着我们通过 <code>strace</code> 可以看到:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>build<span class="w"> </span>--quiet<span class="w"> </span>--release<span class="w"> </span><span class="o">&&</span><span class="w"> </span>strace<span class="w"> </span>-e<span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>./target/release/waytoodeep
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:58:16.425<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">18</span>:58:16.443<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
</code></pre></div>
<p>。。。两个 <code>connect</code> 调用!如我所料!</p>
<blockquote>
<p>谬论:一个 <code>connect</code> 调用都没看到?因为 Rust 构建 HTTP/2 请求的时候甚至都需要建立 TCP 连接。真是革命性的!</p>
</blockquote>
<p>这当然不是真的。可能在其他线程执行了?也许 <code>strace</code> 默认仅跟踪了主线程?</p>
<p>啊,对了, <code>-f</code> 可以跟踪所有「子进程」,就像大家知道的那样 Linux 线程仅仅是披了件风衣的进程(或者其他方式)。所以,让我们看一下:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>build<span class="w"> </span>--quiet<span class="w"> </span>--release<span class="w"> </span><span class="o">&&</span><span class="w"> </span>strace<span class="w"> </span>-f<span class="w"> </span>-e<span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>./target/release/waytoodeep
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154612</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154613</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154614</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154615</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154616</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154617</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154618</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154619</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154620</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154621</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154622</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154623</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154624</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154625</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154626</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154627</span><span class="w"> </span>attached
strace:<span class="w"> </span>Process<span class="w"> </span><span class="m">154628</span><span class="w"> </span>attached
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNIX,<span class="w"> </span><span class="nv">sun_path</span><span class="o">=</span><span class="s2">"/var/run/nscd/socket"</span><span class="o">}</span>,<span class="w"> </span><span class="m">110</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENOENT<span class="w"> </span><span class="o">(</span>No<span class="w"> </span>such<span class="w"> </span>file<span class="w"> </span>or<span class="w"> </span>directory<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNIX,<span class="w"> </span><span class="nv">sun_path</span><span class="o">=</span><span class="s2">"/var/run/nscd/socket"</span><span class="o">}</span>,<span class="w"> </span><span class="m">110</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENOENT<span class="w"> </span><span class="o">(</span>No<span class="w"> </span>such<span class="w"> </span>file<span class="w"> </span>or<span class="w"> </span>directory<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNIX,<span class="w"> </span><span class="nv">sun_path</span><span class="o">=</span><span class="s2">"/var/run/nscd/socket"</span><span class="o">}</span>,<span class="w"> </span><span class="m">110</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENOENT<span class="w"> </span><span class="o">(</span>No<span class="w"> </span>such<span class="w"> </span>file<span class="w"> </span>or<span class="w"> </span>directory<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">53</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"127.0.0.53"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">53</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"127.0.0.53"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET6,<span class="w"> </span><span class="nv">sin6_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_flowinfo</span><span class="o">=</span>htonl<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span>inet_pton<span class="o">(</span>AF_INET6,<span class="w"> </span><span class="s2">"2606:4700:3034::6815:5ca9"</span>,<span class="w"> </span><span class="p">&</span>sin6_addr<span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_scope_id</span><span class="o">=</span><span class="m">0</span><span class="o">}</span>,<span class="w"> </span><span class="m">28</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENETUNREACH<span class="w"> </span><span class="o">(</span>Network<span class="w"> </span>is<span class="w"> </span>unreachable<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET6,<span class="w"> </span><span class="nv">sin6_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_flowinfo</span><span class="o">=</span>htonl<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span>inet_pton<span class="o">(</span>AF_INET6,<span class="w"> </span><span class="s2">"2606:4700:3031::ac43:c490"</span>,<span class="w"> </span><span class="p">&</span>sin6_addr<span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_scope_id</span><span class="o">=</span><span class="m">0</span><span class="o">}</span>,<span class="w"> </span><span class="m">28</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENETUNREACH<span class="w"> </span><span class="o">(</span>Network<span class="w"> </span>is<span class="w"> </span>unreachable<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"104.21.92.169"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"172.67.196.144"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET6,<span class="w"> </span><span class="nv">sin6_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_flowinfo</span><span class="o">=</span>htonl<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span>inet_pton<span class="o">(</span>AF_INET6,<span class="w"> </span><span class="s2">"2606:4700:3034::6815:5ca9"</span>,<span class="w"> </span><span class="p">&</span>sin6_addr<span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_scope_id</span><span class="o">=</span><span class="m">0</span><span class="o">}</span>,<span class="w"> </span><span class="m">28</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENETUNREACH<span class="w"> </span><span class="o">(</span>Network<span class="w"> </span>is<span class="w"> </span>unreachable<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET6,<span class="w"> </span><span class="nv">sin6_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_flowinfo</span><span class="o">=</span>htonl<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span>inet_pton<span class="o">(</span>AF_INET6,<span class="w"> </span><span class="s2">"2606:4700:3031::ac43:c490"</span>,<span class="w"> </span><span class="p">&</span>sin6_addr<span class="o">)</span>,<span class="w"> </span><span class="nv">sin6_scope_id</span><span class="o">=</span><span class="m">0</span><span class="o">}</span>,<span class="w"> </span><span class="m">28</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ENETUNREACH<span class="w"> </span><span class="o">(</span>Network<span class="w"> </span>is<span class="w"> </span>unreachable<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"104.21.92.169"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_UNSPEC,<span class="w"> </span><span class="nv">sa_data</span><span class="o">=</span><span class="s2">"\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span><span class="o">}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">0</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"172.67.196.144"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154625</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"104.21.92.169"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154626</span><span class="o">]</span><span class="w"> </span>connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"104.21.92.169"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:00:53.862<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:00:53.880<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">154628</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154627</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154618</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154614</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154612</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154619</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154617</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154613</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154615</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154623</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154616</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154624</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154621</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154622</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154626</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154620</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
<span class="o">[</span>pid<span class="w"> </span><span class="m">154625</span><span class="o">]</span><span class="w"> </span>+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++shell
</code></pre></div>
<p>哇哦,一大堆 <code>connect</code> 。</p>
<p>所以程序首先尝试连接 <a href="https://jameshfisher.com/2018/02/05/dont-use-nscd/">nscd</a> 因为显然我们依然生活在 90 年代:</p>
<div class="highlight"><pre><span></span><code><span class="p">[</span><span class="n">pid</span><span class="w"> </span><span class="mi">154627</span><span class="p">]</span><span class="w"> </span><span class="n">connect</span><span class="p">(</span><span class="mi">9</span><span class="p">,</span><span class="w"> </span><span class="p">{</span><span class="n">sa_family</span><span class="o">=</span><span class="n">AF_UNIX</span><span class="p">,</span><span class="w"> </span><span class="n">sun_path</span><span class="o">=</span><span class="s2">"/var/run/nscd/socket"</span><span class="p">},</span><span class="w"> </span><span class="mi">110</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="w"> </span><span class="n">ENOENT</span><span class="w"> </span><span class="p">(</span><span class="n">No</span><span class="w"> </span><span class="n">such</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="ow">or</span><span class="w"> </span><span class="n">directory</span><span class="p">)</span>
</code></pre></div>
<p>。。。幸好我的系统没有它,所以它继续通过 <code>/etc/resolv.conf</code> 查询 DNS:</p>
<div class="highlight"><pre><span></span><code>[<span class="nv">pid</span><span class="w"> </span><span class="mi">154628</span>]<span class="w"> </span><span class="k">connect</span><span class="ss">(</span><span class="mi">9</span>,<span class="w"> </span>{<span class="nv">sa_family</span><span class="o">=</span><span class="nv">AF_INET</span>,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span><span class="nv">htons</span><span class="ss">(</span><span class="mi">53</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span><span class="nv">inet_addr</span><span class="ss">(</span><span class="s2">"127.0.0.53"</span><span class="ss">)</span>},<span class="w"> </span><span class="mi">16</span><span class="ss">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
</code></pre></div>
<p>然后最终获得一些 <a href="https://www.cloudflare.com/ips/">Cloudflare 的 IP 地址</a>,如 <code>172.67.196.144</code> 和 <code>104.21.92.169</code> 。还有一些 IPv6 相关的,由于我禁用了 IPv6 所以并没有工作:</p>
<div class="highlight"><pre><span></span><code>[<span class="nv">pid</span><span class="w"> </span><span class="mi">154627</span>]<span class="w"> </span><span class="k">connect</span><span class="ss">(</span><span class="mi">9</span>,<span class="w"> </span>{<span class="nv">sa_family</span><span class="o">=</span><span class="nv">AF_INET6</span>,<span class="w"> </span><span class="nv">sin6_port</span><span class="o">=</span><span class="nv">htons</span><span class="ss">(</span><span class="mi">0</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">sin6_flowinfo</span><span class="o">=</span><span class="nv">htonl</span><span class="ss">(</span><span class="mi">0</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">inet_pton</span><span class="ss">(</span><span class="nv">AF_INET6</span>,<span class="w"> </span><span class="s2">"2606:4700:3034::6815:5ca9"</span>,<span class="w"> </span><span class="o">&</span><span class="nv">sin6_addr</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">sin6_scope_id</span><span class="o">=</span><span class="mi">0</span>},<span class="w"> </span><span class="mi">28</span><span class="ss">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="w"> </span><span class="nv">ENETUNREACH</span><span class="w"> </span><span class="ss">(</span><span class="nv">Network</span><span class="w"> </span><span class="nv">is</span><span class="w"> </span><span class="nv">unreachable</span><span class="ss">)</span>
</code></pre></div>
<p>然后终于程序决定使用 IPv4 的地址 <code>104.21.92.169</code> 去构建请求,同时我们能看到这些都是非阻塞的(non-blocking)连接,因为 <code>connect</code> 返回 <code>-1</code> 而不是 <code>0</code> 表示「正在连接、正在连接、稍后回来检查」。</p>
<div class="highlight"><pre><span></span><code>[<span class="nv">pid</span><span class="w"> </span><span class="mi">154625</span>]<span class="w"> </span><span class="k">connect</span><span class="ss">(</span><span class="mi">9</span>,<span class="w"> </span>{<span class="nv">sa_family</span><span class="o">=</span><span class="nv">AF_INET</span>,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span><span class="nv">htons</span><span class="ss">(</span><span class="mi">443</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span><span class="nv">inet_addr</span><span class="ss">(</span><span class="s2">"104.21.92.169"</span><span class="ss">)</span>},<span class="w"> </span><span class="mi">16</span><span class="ss">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="w"> </span><span class="nv">EINPROGRESS</span><span class="w"> </span><span class="ss">(</span><span class="nv">Operation</span><span class="w"> </span><span class="nv">now</span><span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">progress</span><span class="ss">)</span>
[<span class="nv">pid</span><span class="w"> </span><span class="mi">154626</span>]<span class="w"> </span><span class="k">connect</span><span class="ss">(</span><span class="mi">10</span>,<span class="w"> </span>{<span class="nv">sa_family</span><span class="o">=</span><span class="nv">AF_INET</span>,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span><span class="nv">htons</span><span class="ss">(</span><span class="mi">443</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span><span class="nv">inet_addr</span><span class="ss">(</span><span class="s2">"104.21.92.169"</span><span class="ss">)</span>},<span class="w"> </span><span class="mi">16</span><span class="ss">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="w"> </span><span class="nv">EINPROGRESS</span><span class="w"> </span><span class="ss">(</span><span class="nv">Operation</span><span class="w"> </span><span class="nv">now</span><span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">progress</span><span class="ss">)</span>
</code></pre></div>
<p>好了!所以忽略 <a href="https://isitdns.com/">DNS</a> 的话我们看到了两个连接。</p>
<p>同时我们看到了一些线程。</p>
<p>这就是 Rust 异步的工作方式?我们只是用了一些线程?这也就是它能在「后台运行」的原因?</p>
<p>在我们回答这些问题前,让我们先调整我们的代码真正的去等待 future 完成,而不是随意的休眠 1 秒钟。</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">leaked_client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Box</span>::<span class="n">leak</span><span class="p">(</span><span class="nb">Box</span>::<span class="n">new</span><span class="p">(</span><span class="n">client</span><span class="p">));</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">leaked_client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">leaked_client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">handle1</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">handle2</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>等等,我们这不又回到原点吗?等待第一个请求完成,然后才开始第二个请求。</p>
<p>当然不是!我们运行几次就可以看到:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:07.934<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:07.958<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:08.676<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:08.680<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:09.325<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:09.338<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:10.134<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:11:10.144<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>。。。大部分情况下“whats-in-the-box”胜出了(它确实先开始),但是“advent-of-code-2020”也首先完成了几次。这也是我们希望看到的。</p>
<blockquote>
<p>谬论:也就是说因为有线程请求被并行(parallel)的执行了。</p>
</blockquote>
<p>不是的。但是不要相信我,让我们继续深入。</p>
<h3 id="不是因为线程">不是因为线程</h3>
<p>让我们通过 GDB 运行我们的小程序,大部分原因是我还没有对 LLDB 形成肌肉记忆,我相信这是水到渠成的事。</p>
<div class="highlight"><pre><span></span><code><span class="err">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">build</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">gdb</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">--</span><span class="n">args</span><span class="w"> </span><span class="p">.</span><span class="o">/</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span>
<span class="n">Reading</span><span class="w"> </span><span class="n">symbols</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="p">.</span><span class="o">/</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="p">...</span>
<span class="nl">warning</span><span class="p">:</span><span class="w"> </span><span class="n">Missing</span><span class="w"> </span><span class="n">auto</span><span class="o">-</span><span class="k">load</span><span class="w"> </span><span class="n">script</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="n">offset</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="k">section</span><span class="w"> </span><span class="p">.</span><span class="n">debug_gdb_scripts</span>
<span class="k">of</span><span class="w"> </span><span class="k">file</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="o">/</span><span class="n">target</span><span class="o">/</span><span class="n">debug</span><span class="o">/</span><span class="n">waytoodeep</span><span class="p">.</span>
<span class="k">Use</span><span class="w"> </span><span class="err">`</span><span class="n">info</span><span class="w"> </span><span class="n">auto</span><span class="o">-</span><span class="k">load</span><span class="w"> </span><span class="n">python</span><span class="o">-</span><span class="n">scripts</span><span class="w"> </span><span class="o">[</span><span class="n">REGEXP</span><span class="o">]</span><span class="err">'</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">list</span><span class="w"> </span><span class="n">them</span><span class="p">.</span>
<span class="p">(</span><span class="n">gdb</span><span class="p">)</span>
</code></pre></div>
<p>一切就绪!</p>
<p>在我们开始之前先设置一下断点。我说了断点?应该是捕捉点(catchpoint)。我不知道参与构造 HTTP/2 请求的所有函数名,但是我知道 <code>connect</code> 对应的系统调用(syscall),这也是我们需要打断点的地方,或者捕捉(catch)。</p>
<div class="highlight"><pre><span></span><code><span class="ss">(</span><span class="nv">gdb</span><span class="ss">)</span><span class="w"> </span><span class="nv">catch</span><span class="w"> </span><span class="nv">syscall</span><span class="w"> </span><span class="k">connect</span>
<span class="nv">Catchpoint</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="ss">(</span><span class="nv">syscall</span><span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>[<span class="mi">42</span>]<span class="ss">)</span>
</code></pre></div>
<p>现在我们开始!</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>Starting<span class="w"> </span>program:<span class="w"> </span>/home/amos/ftl/waytoodeep/target/debug/waytoodeep
<span class="o">[</span>Thread<span class="w"> </span>debugging<span class="w"> </span>using<span class="w"> </span>libthread_db<span class="w"> </span>enabled<span class="o">]</span>
Using<span class="w"> </span>host<span class="w"> </span>libthread_db<span class="w"> </span>library<span class="w"> </span><span class="s2">"/lib/x86_64-linux-gnu/libthread_db.so.1"</span>.
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7c28700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158945</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7a27700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158946</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7fffef826700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158947</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7826700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158948</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7625700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158949</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7424700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158950</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff7223700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158951</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff701f700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158952</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff6e1e700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158953</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff6c1a700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158954</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff6a16700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158955</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff680f700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158956</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff660e700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158957</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff640a700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158958</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff6206700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158959</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff5f4b700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158960</span><span class="o">)]</span>
<span class="o">[</span>New<span class="w"> </span>Thread<span class="w"> </span>0x7ffff5d4a700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158961</span><span class="o">)]</span>
<span class="o">[</span>Switching<span class="w"> </span>to<span class="w"> </span>Thread<span class="w"> </span>0x7ffff5f4b700<span class="w"> </span><span class="o">(</span>LWP<span class="w"> </span><span class="m">158960</span><span class="o">)]</span>
Thread<span class="w"> </span><span class="m">17</span><span class="w"> </span><span class="s2">"tokio-runtime-w"</span><span class="w"> </span>hit<span class="w"> </span>Catchpoint<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="o">(</span>call<span class="w"> </span>to<span class="w"> </span>syscall<span class="w"> </span>connect<span class="o">)</span>,<span class="w"> </span>0x00007ffff7d5033b<span class="w"> </span><span class="k">in</span><span class="w"> </span>__libc_connect<span class="w"> </span><span class="o">(</span><span class="nv">fd</span><span class="o">=</span>fd@entry<span class="o">=</span><span class="m">9</span>,<span class="w"> </span><span class="nv">addr</span><span class="o">=</span>...,<span class="w"> </span>addr@entry<span class="o">=</span>...,
<span class="w"> </span><span class="nv">len</span><span class="o">=</span>len@entry<span class="o">=</span><span class="m">110</span><span class="o">)</span><span class="w"> </span>at<span class="w"> </span>../sysdeps/unix/sysv/linux/connect.c:26
<span class="m">26</span><span class="w"> </span>../sysdeps/unix/sysv/linux/connect.c:<span class="w"> </span>No<span class="w"> </span>such<span class="w"> </span>file<span class="w"> </span>or<span class="w"> </span>directory.
<span class="o">(</span>gdb<span class="o">)</span>
</code></pre></div>
<p>不错不错,真快!我们停在了名为 <code>tokio-runtime-w</code> 的 <code>Thread 17</code> 中,因为我猜其他所有字母都被使用了。</p>
<blockquote>
<p><code>w</code> 意味这 <code>worker</code> ,如果你不是第一天用 Unix 就会知道什么这么简写。</p>
</blockquote>
<p>好的, <code>Thread 17</code> ,那么其他线程在做什么呢?</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="n">gdb</span><span class="p">)</span><span class="w"> </span><span class="n">info</span><span class="w"> </span><span class="n">threads</span>
<span class="w"> </span><span class="n">Id</span><span class="w"> </span><span class="n">Target</span><span class="w"> </span><span class="n">Id</span><span class="w"> </span><span class="n">Frame</span>
<span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7c2c6c0</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158941</span><span class="p">)</span><span class="w"> </span><span class="ss">"waytoodeep"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7c28700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158945</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7a27700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158946</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="mh">0x00007ffff7d4f5ce</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">epoll_wait</span><span class="w"> </span><span class="p">(</span><span class="n">epfd</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="w"> </span><span class="n">events</span><span class="o">=</span><span class="mh">0x555556338b60</span><span class="p">,</span><span class="w"> </span><span class="n">maxevents</span><span class="o">=</span><span class="mi">1024</span><span class="p">,</span><span class="w"> </span><span class="n">timeout</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
<span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">epoll_wait</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">30</span>
<span class="w"> </span><span class="mi">4</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7fffef826700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158947</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7826700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158948</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">6</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7625700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158949</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">7</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7424700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158950</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">8</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff7223700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158951</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">9</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff701f700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158952</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6e1e700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158953</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">11</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6c1a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158954</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">12</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6a16700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158955</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">13</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff680f700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158956</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">14</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff660e700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158957</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">15</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff640a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158958</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6206700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158959</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="w"> </span><span class="o">*</span><span class="mi">17</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff5f4b700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158960</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="mh">0x00007ffff7d5033b</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">__libc_connect</span><span class="w"> </span><span class="p">(</span><span class="n">fd</span><span class="o">=</span><span class="n">fd</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">9</span><span class="p">,</span><span class="w"> </span><span class="n">addr</span><span class="o">=</span><span class="p">...,</span><span class="w"> </span><span class="n">addr</span><span class="nv">@entry</span><span class="o">=</span><span class="p">...,</span><span class="w"> </span><span class="nf">len</span><span class="o">=</span><span class="nf">len</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">110</span><span class="p">)</span>
<span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="k">connect</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">26</span>
<span class="w"> </span><span class="mi">18</span><span class="w"> </span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff5d4a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158961</span><span class="p">)</span><span class="w"> </span><span class="ss">"tokio-runtime-w"</span><span class="w"> </span><span class="mh">0x00007ffff7d48a46</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">__GI___mmap64</span><span class="w"> </span><span class="p">(</span><span class="n">offset</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">fd</span><span class="o">=-</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="n">flags</span><span class="o">=</span><span class="mi">16418</span><span class="p">,</span><span class="w"> </span><span class="n">prot</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="nf">len</span><span class="o">=</span><span class="mi">134217728</span><span class="p">,</span><span class="w"> </span><span class="n">addr</span><span class="o">=</span><span class="mh">0x0</span><span class="p">)</span>
<span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">mmap64</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">59</span>
</code></pre></div>
<p>额。</p>
<p>我们可以获得更多的栈帧?</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="n">gdb</span><span class="p">)</span><span class="w"> </span><span class="n">thread</span><span class="w"> </span><span class="n">apply</span><span class="w"> </span><span class="ow">all</span><span class="w"> </span><span class="n">backtrace</span><span class="w"> </span><span class="mi">2</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">18</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff5d4a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158961</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="mh">0x00007ffff7d48a46</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">__GI___mmap64</span><span class="w"> </span><span class="p">(</span><span class="n">offset</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">fd</span><span class="o">=-</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="n">flags</span><span class="o">=</span><span class="mi">16418</span><span class="p">,</span><span class="w"> </span><span class="n">prot</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="nf">len</span><span class="o">=</span><span class="mi">134217728</span><span class="p">,</span><span class="w"> </span><span class="n">addr</span><span class="o">=</span><span class="mh">0x0</span><span class="p">)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">mmap64</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">59</span>
<span class="n">#1</span><span class="w"> </span><span class="n">__GI___mmap64</span><span class="w"> </span><span class="p">(</span><span class="n">addr</span><span class="o">=</span><span class="n">addr</span><span class="nv">@entry</span><span class="o">=</span><span class="mh">0x0</span><span class="p">,</span><span class="w"> </span><span class="nf">len</span><span class="o">=</span><span class="nf">len</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">134217728</span><span class="p">,</span><span class="w"> </span><span class="n">prot</span><span class="o">=</span><span class="n">prot</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">flags</span><span class="o">=</span><span class="n">flags</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">16418</span><span class="p">,</span><span class="w"> </span><span class="n">fd</span><span class="o">=</span><span class="n">fd</span><span class="nv">@entry</span><span class="o">=-</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="n">offset</span><span class="o">=</span><span class="n">offset</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">mmap64</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">47</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">17</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff5f4b700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158960</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="mh">0x00007ffff7d5033b</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">__libc_connect</span><span class="w"> </span><span class="p">(</span><span class="n">fd</span><span class="o">=</span><span class="n">fd</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">9</span><span class="p">,</span><span class="w"> </span><span class="n">addr</span><span class="o">=</span><span class="p">...,</span><span class="w"> </span><span class="n">addr</span><span class="nv">@entry</span><span class="o">=</span><span class="p">...,</span><span class="w"> </span><span class="nf">len</span><span class="o">=</span><span class="nf">len</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">110</span><span class="p">)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="k">connect</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">26</span>
<span class="n">#1</span><span class="w"> </span><span class="mh">0x00007ffff7d8b713</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">open_socket</span><span class="w"> </span><span class="p">(</span><span class="n">type</span><span class="o">=</span><span class="n">type</span><span class="nv">@entry</span><span class="o">=</span><span class="n">GETFDHST</span><span class="p">,</span><span class="w"> </span><span class="k">key</span><span class="o">=</span><span class="k">key</span><span class="nv">@entry</span><span class="o">=</span><span class="mh">0x7ffff7de5ccb</span><span class="w"> </span><span class="ss">"hosts"</span><span class="p">,</span><span class="w"> </span><span class="n">keylen</span><span class="o">=</span><span class="n">keylen</span><span class="nv">@entry</span><span class="o">=</span><span class="mi">6</span><span class="p">)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="n">nscd_helper</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">185</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff6206700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158959</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="n">#1</span><span class="w"> </span><span class="mh">0x0000555555b9f1d1</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="nl">parking_lot_core</span><span class="p">:</span><span class="err">:</span><span class="nl">thread_parker</span><span class="p">:</span><span class="err">:</span><span class="nl">imp</span><span class="p">:</span><span class="err">:</span><span class="nl">ThreadParker</span><span class="p">:</span><span class="err">:</span><span class="n">futex_wait</span><span class="w"> </span><span class="p">(</span><span class="n">self</span><span class="o">=</span><span class="mh">0x7ffff6206498</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="o">=</span><span class="p">...)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="p">.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="p">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">parking_lot_core</span><span class="o">-</span><span class="mf">0.8.3</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread_parker</span><span class="o">/</span><span class="n">linux</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">112</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">15</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff640a700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158958</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="n">#1</span><span class="w"> </span><span class="mh">0x0000555555b9f1d1</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="nl">parking_lot_core</span><span class="p">:</span><span class="err">:</span><span class="nl">thread_parker</span><span class="p">:</span><span class="err">:</span><span class="nl">imp</span><span class="p">:</span><span class="err">:</span><span class="nl">ThreadParker</span><span class="p">:</span><span class="err">:</span><span class="n">futex_wait</span><span class="w"> </span><span class="p">(</span><span class="n">self</span><span class="o">=</span><span class="mh">0x7ffff640a498</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="o">=</span><span class="p">...)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="p">.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="p">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">parking_lot_core</span><span class="o">-</span><span class="mf">0.8.3</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread_parker</span><span class="o">/</span><span class="n">linux</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">112</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">14</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff660e700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158957</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="n">#1</span><span class="w"> </span><span class="mh">0x0000555555b9f1d1</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="nl">parking_lot_core</span><span class="p">:</span><span class="err">:</span><span class="nl">thread_parker</span><span class="p">:</span><span class="err">:</span><span class="nl">imp</span><span class="p">:</span><span class="err">:</span><span class="nl">ThreadParker</span><span class="p">:</span><span class="err">:</span><span class="n">futex_wait</span><span class="w"> </span><span class="p">(</span><span class="n">self</span><span class="o">=</span><span class="mh">0x7ffff660e498</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="o">=</span><span class="p">...)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="p">.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="p">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">parking_lot_core</span><span class="o">-</span><span class="mf">0.8.3</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread_parker</span><span class="o">/</span><span class="n">linux</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">112</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
<span class="n">Thread</span><span class="w"> </span><span class="mi">13</span><span class="w"> </span><span class="p">(</span><span class="n">Thread</span><span class="w"> </span><span class="mh">0x7ffff680f700</span><span class="w"> </span><span class="p">(</span><span class="n">LWP</span><span class="w"> </span><span class="mi">158956</span><span class="p">))</span><span class="err">:</span>
<span class="n">#0</span><span class="w"> </span><span class="n">syscall</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="p">..</span><span class="o">/</span><span class="n">sysdeps</span><span class="o">/</span><span class="n">unix</span><span class="o">/</span><span class="n">sysv</span><span class="o">/</span><span class="n">linux</span><span class="o">/</span><span class="n">x86_64</span><span class="o">/</span><span class="n">syscall</span><span class="p">.</span><span class="nl">S</span><span class="p">:</span><span class="mi">38</span>
<span class="n">#1</span><span class="w"> </span><span class="mh">0x0000555555b9f1d1</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="nl">parking_lot_core</span><span class="p">:</span><span class="err">:</span><span class="nl">thread_parker</span><span class="p">:</span><span class="err">:</span><span class="nl">imp</span><span class="p">:</span><span class="err">:</span><span class="nl">ThreadParker</span><span class="p">:</span><span class="err">:</span><span class="n">futex_wait</span><span class="w"> </span><span class="p">(</span><span class="n">self</span><span class="o">=</span><span class="mh">0x7ffff680f498</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="o">=</span><span class="p">...)</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="p">.</span><span class="n">cargo</span><span class="o">/</span><span class="n">registry</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">github</span><span class="p">.</span><span class="n">com</span><span class="o">-</span><span class="mi">1</span><span class="n">ecc6299db9ec823</span><span class="o">/</span><span class="n">parking_lot_core</span><span class="o">-</span><span class="mf">0.8.3</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">thread_parker</span><span class="o">/</span><span class="n">linux</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">112</span>
<span class="p">(</span><span class="n">More</span><span class="w"> </span><span class="n">stack</span><span class="w"> </span><span class="n">frames</span><span class="w"> </span><span class="n">follow</span><span class="p">...)</span>
</code></pre></div>
<p>额。大部分都是挂起的。也就是空闲的。更准确的是它们在等待工作。</p>
<p>我们也可以通过 htop 查看这些所有线程,我知道我们已经看到了,但是我仅仅是觉得 htop 很棒。感谢 <a href="https://twitter.com/hisham%5Fhm">Hisham</a>!
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/htop-colors.571c5effbff8a0b3.webp">
所以,我们注意到一些线程,同时也有一些 CPU 核心。可能是一个 CPU 核心一个线程?
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/worker-threads.64dbe39e33ccfc4f.webp">
是的。然后还有一些阻塞的线程,正如我们从上面 <code>strace</code> 输出看到的那样, 它会进行一些阻塞的 <code>connect</code> 调用去查询 DNS(实际是 glibc 在执行),
所以它通过运行在工作线程之外避免阻塞其他任务。</p>
<blockquote>
<p>所以多个线程,这就是为什么一次可以运行多个请求的原因?</p>
</blockquote>
<p>实际上文档上表明这是一个单线程的执行器,我也不能确定,所以让我们试一下:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// 👇</span>
<span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// (same as before)</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:50:15.977<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">19</span>:50:15.994<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>两个响应间隔 <code>17ms</code> ,这个时间不够构造一个完整的请求,所以请求并行(parallel)的执行了。如果你依然坚持它内部使用了线程,让我们进一步确认我们只有一个线程:
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/current-thread.cd7b619ed644899b.webp">
确实有多个线程,但是这些都是阻塞线程。仅仅是 DNS 查询。可以通过 htop 看到已经没有无数(15)的工作线程了:
<img alt="" src="https://fasterthanli.me/content/articles/understanding-rust-futures-by-going-way-too-deep/assets/htop-current-thread.fe28174abc5d15fa.webp">
(顺便说一下 15 个工作线程的原因,这是因为我预留了一个 CPU 核心没有分配给虚拟机,这样即使虚拟机全速运行也不会导致宿主机停止响应)。</p>
<p>如果我们将 DNS 查询排除在外,我们就可以看到实际上仅仅使用了一个线程,我们将继续下去,以防你依然存疑!</p>
<h3 id="插曲-让我们避免泄漏内存">插曲:让我们避免泄漏内存</h3>
<p>但是在那之前:正在泄漏 reqwest 的 <code>Client</code> 让我很不爽。</p>
<p>为了避免,我们可以创建一个原子引用计数(atomically-reference-counted),这样它就可以随着任务运行而存活。</p>
<p>修改起来非常简单:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// 👇 Atomically Reference Counted = Arc</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">sync</span>::<span class="n">Arc</span><span class="p">;</span>
<span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// 👇 there we go</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Arc</span>::<span class="n">new</span><span class="p">(</span><span class="n">Client</span>::<span class="n">new</span><span class="p">());</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">.</span><span class="n">clone</span><span class="p">(),</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// (cloning it only increases the reference count)</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">.</span><span class="n">clone</span><span class="p">(),</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">handle1</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">handle2</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="cp">#[allow(clippy::manual_async_fn)]</span>
<span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span>
<span class="w"> </span><span class="c1">// 👇 now taking this, we have shared ownership of it</span>
<span class="w"> </span><span class="n">client</span>: <span class="nc">Arc</span><span class="o"><</span><span class="n">Client</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="nb">static</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// luckily this 👇 only requires `&self`</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>好了,现在我感觉好多了。我们的程序不再泄漏一些字节即使它永远不会运行超过几秒钟。一切都还好。</p>
<p>让我们看一下 <code>reqwest</code> 的 <code>Client</code> 定义:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[derive(Clone)]</span>
<span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">Client</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">inner</span>: <span class="nc">Arc</span><span class="o"><</span><span class="n">ClientRef</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>它已经是引用计数的了,所以我们可以直接接受一个 <code>Client</code>:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">.</span><span class="n">clone</span><span class="p">(),</span><span class="w"> </span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// no need to clone a second time</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">handle1</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">handle2</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="cp">#[allow(clippy::manual_async_fn)]</span>
<span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">client</span>: <span class="nc">Client</span><span class="p">,</span>
<span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="o">'</span><span class="nb">static</span> <span class="kt">str</span><span class="p">,</span>
<span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">'</span><span class="nb">static</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>好了。</p>
<p>对了,仅供参考,更简单的 <code>async fn</code> 也可以工作了:</p>
<div class="highlight"><pre><span></span><code><span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">client</span>: <span class="nc">Client</span><span class="p">,</span><span class="w"> </span><span class="n">url</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">send</span><span class="p">().</span><span class="k">await</span><span class="o">?</span><span class="p">.</span><span class="n">error_for_status</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">content_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">?</span><span class="n">res</span><span class="p">.</span><span class="n">headers</span><span class="p">().</span><span class="n">get</span><span class="p">(</span><span class="s">"content-type"</span><span class="p">),</span><span class="w"> </span><span class="s">"Got a response!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>我们甚至不需要要求 <code>url</code> 的借用生命周期是 <code>'static</code> 。如果 <code>url</code> 是 <code>'static</code> 的则返回的 Future 也是,反之亦然。</p>
<p>作为例子,下面代码无法通过编译:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="c1">// this is a `String`, owned by main</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">url1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">String</span>::<span class="n">from</span><span class="p">(</span><span class="n">URL_1</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// we're borrowing from main 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">.</span><span class="n">clone</span><span class="p">(),</span><span class="w"> </span><span class="o">&</span><span class="n">url1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">fut2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut1</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">handle2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">spawn</span><span class="p">(</span><span class="n">fut2</span><span class="p">);</span>
<span class="w"> </span><span class="n">handle1</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">handle2</span><span class="p">.</span><span class="k">await</span><span class="p">.</span><span class="n">unwrap</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="k">check</span>
<span class="w"> </span><span class="n">Checking</span><span class="w"> </span><span class="n">waytoodeep</span><span class="w"> </span><span class="n">v0</span><span class="mf">.1.0</span><span class="w"> </span><span class="p">(</span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="n">ftl</span><span class="o">/</span><span class="n">waytoodeep</span><span class="p">)</span>
<span class="k">error</span><span class="err">[</span><span class="n">E0597</span><span class="err">]</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`url1`</span><span class="w"> </span><span class="n">does</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="n">live</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">enough</span>
<span class="w"> </span><span class="o">--></span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">18</span><span class="o">:</span><span class="mi">44</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">18</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">let</span><span class="w"> </span><span class="n">fut1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="k">client</span><span class="p">.</span><span class="k">clone</span><span class="p">(),</span><span class="w"> </span><span class="o">&</span><span class="n">url1</span><span class="p">);</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">----------------------------^^^^^-</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">borrowed</span><span class="w"> </span><span class="k">value</span><span class="w"> </span><span class="n">does</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="n">live</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">enough</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">argument</span><span class="w"> </span><span class="n">requires</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n n-Quoted">`url1`</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">borrowed</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n n-Quoted">`'static`</span>
<span class="p">...</span>
<span class="mi">28</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n n-Quoted">`url1`</span><span class="w"> </span><span class="n">dropped</span><span class="w"> </span><span class="n">here</span><span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="n">still</span><span class="w"> </span><span class="n">borrowed</span>
</code></pre></div>
<blockquote>
<p>你面对的考验就是:修改了一些代码,然后突然间整个 <code>Future</code> 不再实现 <code>Send</code> ,但是你需要它实现 <code>Send</code> 。参考<a href="https://fasterthanli.me/articles/getting-in-and-out-of-trouble-with-rust-futures">Getting in and out of trouble with Rust futures</a>。</p>
</blockquote>
<p>在我们进一步深入之前,我们还想提一下,除了通过 <code>tokio::spawn</code> 可以同时运行两个 future 并且立即等待两个 future 完成,还是使用 <code>FuturesUnordered</code> 实现相同目的。</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">futures</span><span class="mf">@0.3.16</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">futures</span><span class="w"> </span><span class="n">v0</span><span class="mf">.3.16</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">futures</span>::<span class="p">{</span><span class="n">stream</span>::<span class="n">FuturesUnordered</span><span class="p">,</span><span class="w"> </span><span class="n">StreamExt</span><span class="p">};</span>
<span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">client</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Client</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">group</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="fm">vec!</span><span class="p">[</span>
<span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">.</span><span class="n">clone</span><span class="p">(),</span><span class="w"> </span><span class="n">URL_1</span><span class="p">),</span>
<span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="n">client</span><span class="p">,</span><span class="w"> </span><span class="n">URL_2</span><span class="p">),</span>
<span class="w"> </span><span class="p">]</span>
<span class="w"> </span><span class="p">.</span><span class="n">into_iter</span><span class="p">()</span>
<span class="w"> </span><span class="p">.</span><span class="n">collect</span>::<span class="o"><</span><span class="n">FuturesUnordered</span><span class="o"><</span><span class="n">_</span><span class="o">>></span><span class="p">();</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">item</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">group</span><span class="p">.</span><span class="n">next</span><span class="p">().</span><span class="k">await</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// propagate errors</span>
<span class="w"> </span><span class="n">item</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>通过这个解决半啃啊,我们可以 await 任意数量的 future 对象,同时也是并发的被轮询(polled)。</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:12:37.208<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/articles/whats-in-the-box<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:12:37.227<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>a<span class="w"> </span>response!<span class="w"> </span><span class="nv">url</span><span class="o">=</span>https://fasterthanli.me/series/advent-of-code-2020/part-13<span class="w"> </span><span class="nv">content_type</span><span class="o">=</span>Some<span class="o">(</span><span class="s2">"text/html; charset=utf-8"</span><span class="o">)</span>
</code></pre></div>
<p>仅仅。。。19 毫秒的间隔 -- 可以确定是并发的。</p>
<h3 id="彻底摆脱-dns">彻底摆脱 DNS</h3>
<p>现在让我们暂时忘掉 <code>reqwest</code> 。</p>
<p>HTTP <a href="https://fasterthanli.me/articles/aiming-for-correctness-with-types">并不难</a> ,我们可以自己构建。只要 TCP 就行:</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">net</span>::<span class="n">SocketAddr</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tokio</span>::<span class="p">{</span>
<span class="w"> </span><span class="n">io</span>::<span class="p">{</span><span class="n">AsyncReadExt</span><span class="p">,</span><span class="w"> </span><span class="n">AsyncWriteExt</span><span class="p">},</span>
<span class="w"> </span><span class="n">net</span>::<span class="n">TcpStream</span><span class="p">,</span>
<span class="p">};</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">name</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// look mom, no DNS!</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">addr</span>: <span class="nc">SocketAddr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">],</span><span class="w"> </span><span class="mi">80</span><span class="p">).</span><span class="n">into</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">socket</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">TcpStream</span>::<span class="n">connect</span><span class="p">(</span><span class="n">addr</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// we're writing straight to the socket, there's no buffering</span>
<span class="w"> </span><span class="c1">// so no need to flush</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"GET / HTTP/1.1</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Host: 1.1.1.1</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"User-Agent: cool-bear</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Connection: close</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">String</span>::<span class="n">with_capacity</span><span class="p">(</span><span class="mi">256</span><span class="p">);</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">read_to_string</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">status</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">response</span><span class="p">.</span><span class="n">lines</span><span class="p">().</span><span class="n">next</span><span class="p">().</span><span class="n">unwrap_or_default</span><span class="p">();</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">status</span><span class="p">,</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="s">"Got response!"</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// dropping the socket will close the connection</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>可以正常运行:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:24:05.158<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">301</span><span class="w"> </span>Moved<span class="w"> </span>Permanently<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:24:05.159<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">301</span><span class="w"> </span>Moved<span class="w"> </span>Permanently<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
</code></pre></div>
<p>(看,第二个赢得了胜利)。</p>
<p>同时再也没有 DNS 查询了。</p>
<p>当然 <code>http://1.1.1.1</code> 将我们重定向到 HTTPS 的页面,技术上实现 TLS 并不困难,但是我们的篇幅已经很长了,所以。。。</p>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">tokio</span><span class="o">-</span><span class="n">rustls</span><span class="mf">@0.22.0</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">tokio</span><span class="o">-</span><span class="n">rustls</span><span class="w"> </span><span class="n">v0</span><span class="mf">.22.0</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
<span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">webpki</span><span class="mf">@0.21.4</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">webpki</span><span class="w"> </span><span class="n">v0</span><span class="mf">.21.4</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
<span class="n">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="n">webpki</span><span class="o">-</span><span class="n">roots</span><span class="mf">@0.21.1</span>
<span class="w"> </span><span class="n">Updating</span><span class="w"> </span><span class="err">'</span><span class="n">https</span><span class="o">:</span><span class="c1">//github.com/rust-lang/crates.io-index' index</span>
<span class="w"> </span><span class="n">Adding</span><span class="w"> </span><span class="n">webpki</span><span class="o">-</span><span class="n">roots</span><span class="w"> </span><span class="n">v0</span><span class="mf">.21.1</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">dependencies</span>
</code></pre></div>
<p>然后。。。</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>rm<span class="w"> </span>reqwest
<span class="w"> </span>Removing<span class="w"> </span>reqwest<span class="w"> </span>from<span class="w"> </span>dependencies
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">sync</span>::<span class="n">Arc</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">webpki</span>::<span class="n">DNSNameRef</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tokio_rustls</span>::<span class="p">{</span><span class="n">rustls</span>::<span class="n">ClientConfig</span><span class="p">,</span><span class="w"> </span><span class="n">TlsConnector</span><span class="p">};</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">name</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// look out it's port 443 now</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">addr</span>: <span class="nc">SocketAddr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">],</span><span class="w"> </span><span class="mi">443</span><span class="p">).</span><span class="n">into</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">socket</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">TcpStream</span>::<span class="n">connect</span><span class="p">(</span><span class="n">addr</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// establish a TLS session...</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">connector</span>: <span class="nc">TlsConnector</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">config</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ClientConfig</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="n">config</span>
<span class="w"> </span><span class="p">.</span><span class="n">root_store</span>
<span class="w"> </span><span class="p">.</span><span class="n">add_server_trust_anchors</span><span class="p">(</span><span class="o">&</span><span class="n">webpki_roots</span>::<span class="n">TLS_SERVER_ROOTS</span><span class="p">);</span>
<span class="w"> </span><span class="n">Arc</span>::<span class="n">new</span><span class="p">(</span><span class="n">config</span><span class="p">).</span><span class="n">into</span><span class="p">()</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// we have to use the proper DNS name now 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">dnsname</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DNSNameRef</span>::<span class="n">try_from_ascii_str</span><span class="p">(</span><span class="s">"one.one.one.one"</span><span class="p">)</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">socket</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">connector</span><span class="p">.</span><span class="n">connect</span><span class="p">(</span><span class="n">dnsname</span><span class="p">,</span><span class="w"> </span><span class="n">socket</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// we're writing straight to the socket, there's no buffering</span>
<span class="w"> </span><span class="c1">// so no need to flush</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"GET / HTTP/1.1</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Host: one.one.one.one</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"User-Agent: cool-bear</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Connection: close</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">String</span>::<span class="n">with_capacity</span><span class="p">(</span><span class="mi">256</span><span class="p">);</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">read_to_string</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">status</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">response</span><span class="p">.</span><span class="n">lines</span><span class="p">().</span><span class="n">next</span><span class="p">().</span><span class="n">unwrap_or_default</span><span class="p">();</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">status</span><span class="p">,</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="s">"Got response!"</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// dropping the socket will close the connection</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:31:32.627<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:31:32.658<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
</code></pre></div>
<p>好了,现在返回状态码 200!</p>
<p>我们的目标是了解 Rust 的 future,我们已经获得了不错的进展。</p>
<p>但是让我们考虑以下场景:我们想并发的执行两个请求,一旦其中一个失败,另外一个也应该立即请求失败,或者两个一起成功。</p>
<h3 id="tokio-的-try-join-宏">tokio 的 try_join 宏</h3>
<p>实际上,又一个宏可以做这个!</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">try_join</span><span class="o">!</span><span class="p">(</span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"first"</span><span class="p">),</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"second"</span><span class="p">),)</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">?</span><span class="n">res</span><span class="p">,</span><span class="w"> </span><span class="s">"All done!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>这就是我们想要的!</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:44:52.150<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:44:52.165<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:44:52.165<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(()</span>,<span class="w"> </span><span class="o">())</span>
</code></pre></div>
<p>再次快速检查以下:响应间隔在 15ms -- 也就是确定是并发的发送。</p>
<p><code>try_join!</code> 帮我们进行了 <code>await</code> ,同时帮我们处理了结果。如果一切正常,我们得到所有 future 对象的结果:内容为 <code>Ok</code> 的空元组(有序的)。</p>
<p>所以我们可以取到我们 future 返回的对象:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// 👇</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">name</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><&</span><span class="kt">str</span><span class="p">,</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// (omitted)</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div>
<p>为了方便我们自己,它们按照顺序返回,无论哪个先被执行:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:56.967<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:56.967<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:56.967<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:57.933<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:57.935<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:57.935<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:58.942<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:58.946<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:47:58.946<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
</code></pre></div>
<p>好了,现在我们没有 DNS 查询,我们就可以消除“同时”请求是由于多线程实现的。</p>
<p>因为,如果我们在 <code>strace</code> 下运行程序,并通过 <code>-f</code> 请求跟踪子线程( BTW <code>f</code> 意思是跟踪( <code>follow</code> )子线程):</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cargo<span class="w"> </span>build<span class="w"> </span>--quiet<span class="w"> </span>--release<span class="w"> </span><span class="o">&&</span><span class="w"> </span>strace<span class="w"> </span>-f<span class="w"> </span>-e<span class="w"> </span><span class="s1">'connect'</span><span class="w"> </span>./target/release/waytoodeep
connect<span class="o">(</span><span class="m">9</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"1.1.1.1"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
connect<span class="o">(</span><span class="m">10</span>,<span class="w"> </span><span class="o">{</span><span class="nv">sa_family</span><span class="o">=</span>AF_INET,<span class="w"> </span><span class="nv">sin_port</span><span class="o">=</span>htons<span class="o">(</span><span class="m">443</span><span class="o">)</span>,<span class="w"> </span><span class="nv">sin_addr</span><span class="o">=</span>inet_addr<span class="o">(</span><span class="s2">"1.1.1.1"</span><span class="o">)}</span>,<span class="w"> </span><span class="m">16</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>EINPROGRESS<span class="w"> </span><span class="o">(</span>Operation<span class="w"> </span>now<span class="w"> </span><span class="k">in</span><span class="w"> </span>progress<span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:51:54.004<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:51:54.013<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">20</span>:51:54.015<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
+++<span class="w"> </span>exited<span class="w"> </span>with<span class="w"> </span><span class="m">0</span><span class="w"> </span>+++
</code></pre></div>
<p>。。。现在我们看到了预期的两次 <code>connect</code> 调用,但是没有任何子线程。而且在这个运行中,响应之间的间隔时间是 9 毫秒!少于我直接 ping 1.1.1.1:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>ping<span class="w"> </span>-c<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="m">1</span>.1.1.1
PING<span class="w"> </span><span class="m">1</span>.1.1.1<span class="w"> </span><span class="o">(</span><span class="m">1</span>.1.1.1<span class="o">)</span><span class="w"> </span><span class="m">56</span><span class="o">(</span><span class="m">84</span><span class="o">)</span><span class="w"> </span>bytes<span class="w"> </span>of<span class="w"> </span>data.
<span class="m">64</span><span class="w"> </span>bytes<span class="w"> </span>from<span class="w"> </span><span class="m">1</span>.1.1.1:<span class="w"> </span><span class="nv">icmp_seq</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">ttl</span><span class="o">=</span><span class="m">57</span><span class="w"> </span><span class="nv">time</span><span class="o">=</span><span class="m">13</span>.7<span class="w"> </span>ms
---<span class="w"> </span><span class="m">1</span>.1.1.1<span class="w"> </span>ping<span class="w"> </span>statistics<span class="w"> </span>---
<span class="m">1</span><span class="w"> </span>packets<span class="w"> </span>transmitted,<span class="w"> </span><span class="m">1</span><span class="w"> </span>received,<span class="w"> </span><span class="m">0</span>%<span class="w"> </span>packet<span class="w"> </span>loss,<span class="w"> </span><span class="nb">time</span><span class="w"> </span>0ms
rtt<span class="w"> </span>min/avg/max/mdev<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">13</span>.748/13.748/13.748/0.000<span class="w"> </span>ms
</code></pre></div>
<p>这是因为执行器通过 Event Loop 构建非阻塞的系统调用,然后订阅 Event Loop 管理的资源相关的事件,
然后就可以知道一个 socket 什么时间可以进行读写。</p>
<p>所以,future 对象只是一些状态,接下来就可以进行 await,那么在哪订阅的事件呢?</p>
<p>让我们尝试创建一个我们自己的 <code>try_join</code> -- 一个函数,并且只接受两个 future。然后我们就能看到发生了什么。</p>
<p>我们已经实现了自己的 future,实现一个 <code>try_join</code> 函数会有多麻烦?</p>
<h3 id="事实证明很麻烦">事实证明很麻烦</h3>
<p>我们先从简单的开始!我们想实现一个函数接受两个 future 对象然后返回一个 future 对象。</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/main.rs`</span>
<span class="k">mod</span> <span class="nn">tj</span><span class="p">;</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/tj.rs`</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="n">future</span>::<span class="n">Future</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">()</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">(</span><span class="s">"implement me!"</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>额。函数不应该只返回一个空元组,它需要返回一个包含成功结果的元组。或者遇到的第一个错误。</p>
<p>所以我们需要添加一些更多的范型参数:一个错误类型(我们假设两个 future 对象返回同样的错误类型),另一个是 future 对象返回的 <code>Ok</code> 的类型。</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">(</span><span class="s">"implement me!"</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>好了!这非常绕口,但是我相信我们已经实现了需求。</p>
<p>需要注意的是我们使用了 <code>impl Trait</code> 语法,让我们不用暴露我们自己的 <code>try join future</code> 。这不重要,但是可以让我们用更少的 <code>pub</code> 关键字,同时我们的手指已经码累了。非常累。</p>
<p>所以,让我们来创建这个类型!</p>
<p>类型需要持续 <code>A</code> 和 <code>B</code> ,并注意 <code>AR</code> 、 <code>BR</code> 和 <code>E</code> 类型。所以,希望您对这些范型参数沙拉有个好胃口。</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>然后可以在我们的 <code>try_join</code> 函数中返回它:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="c1">// so simple!</span>
<span class="w"> </span><span class="n">TryJoin</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>我认为这很好的说明一个事实:创建 future 对象仅仅是构建状态。不需要任何额外的工作。</p>
<p>当然,这个不会通过编译,因为 <code>TryJoin</code> 还没有实现 <code>Future</code> 。</p>
<p>但是不要担心! <code>rust-analyzer</code> 可以帮助我们生成缺失的部分:</p>
<div class="highlight"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="p">{</span>
<span class="w"> </span><span class="n">future</span>::<span class="n">Future</span><span class="p">,</span>
<span class="w"> </span><span class="n">pin</span>::<span class="n">Pin</span><span class="p">,</span>
<span class="w"> </span><span class="n">task</span>::<span class="p">{</span><span class="n">Context</span><span class="p">,</span><span class="w"> </span><span class="n">Poll</span><span class="p">},</span>
<span class="p">};</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>如果我们真正的实现了,我们将按照下面方式使用:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tj</span>::<span class="n">try_join</span><span class="p">(</span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"first"</span><span class="p">),</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"second"</span><span class="p">)).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">?</span><span class="n">res</span><span class="p">,</span><span class="w"> </span><span class="s">"All done!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>当然,现在只是会崩溃:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
The<span class="w"> </span>application<span class="w"> </span>panicked<span class="w"> </span><span class="o">(</span>crashed<span class="o">)</span>.
Message:<span class="w"> </span>not<span class="w"> </span>yet<span class="w"> </span>implemented
Location:<span class="w"> </span>src/tj.rs:32
Backtrace<span class="w"> </span>omitted.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>environment<span class="w"> </span>variable<span class="w"> </span>to<span class="w"> </span>display<span class="w"> </span>it.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span>full<span class="w"> </span>to<span class="w"> </span>include<span class="w"> </span><span class="nb">source</span><span class="w"> </span>snippets.
</code></pre></div>
<p>所以,我猜我们需要实现它!</p>
<p>好吧,让我们先尝试至少轮询(polling)一个 future 对象。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">);</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">$</span><span class="w"> </span><span class="n">RUST_LOG</span><span class="o">=</span><span class="n">info</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">--</span><span class="k">release</span>
<span class="k">error</span><span class="err">[</span><span class="n">E0599</span><span class="err">]</span><span class="o">:</span><span class="w"> </span><span class="k">no</span><span class="w"> </span><span class="n">method</span><span class="w"> </span><span class="n">named</span><span class="w"> </span><span class="n n-Quoted">`poll`</span><span class="w"> </span><span class="k">found</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="n">parameter</span><span class="w"> </span><span class="n n-Quoted">`A`</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="k">current</span><span class="w"> </span><span class="n">scope</span>
<span class="w"> </span><span class="o">--></span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">tj</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">32</span><span class="o">:</span><span class="mi">24</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">32</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">self</span><span class="p">.</span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">);</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">^^^^</span><span class="w"> </span><span class="n">method</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">found</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n n-Quoted">`A`</span>
<span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="o">:::</span><span class="w"> </span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">amos</span><span class="o">/</span><span class="p">.</span><span class="n">rustup</span><span class="o">/</span><span class="n">toolchains</span><span class="o">/</span><span class="n">stable</span><span class="o">-</span><span class="n">x86_64</span><span class="o">-</span><span class="no">unknown</span><span class="o">-</span><span class="n">linux</span><span class="o">-</span><span class="n">gnu</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">rustlib</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">rust</span><span class="o">/</span><span class="n">library</span><span class="o">/</span><span class="n">core</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">future</span><span class="o">/</span><span class="n">future</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">100</span><span class="o">:</span><span class="mi">8</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">100</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">fn</span><span class="w"> </span><span class="n">poll</span><span class="p">(</span><span class="n">self</span><span class="o">:</span><span class="w"> </span><span class="n">Pin</span><span class="o"><&</span><span class="n">mut</span><span class="w"> </span><span class="n">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="n">mut</span><span class="w"> </span><span class="k">Context</span><span class="o"><</span><span class="s1">'_>) -> Poll<Self::Output>;</span>
<span class="s1"> | ---- the method is available for `Pin<&mut A>` here</span>
<span class="s1"> |</span>
<span class="s1">help: consider wrapping the receiver expression with the appropriate type</span>
<span class="s1"> |</span>
<span class="s1">32 | let a = Pin::new(&mut self.a).poll(cx);</span>
<span class="s1"> | ^^^^^^^^^^^^^ ^</span>
</code></pre></div>
<p>额!一个好的开始,好的开始。</p>
<p>我已经在这里<a href="https://fasterthanli.me/articles/pin-and-suffering">详细的解释了</a> Pin,所以这里我们就简单的介绍一下。</p>
<p>方法通常通过如下方式定义接收者(receiver):</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">MyType</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">do_thing</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"my value is {}"</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">value</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>也就是下面的简写:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">MyType</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">do_thing</span><span class="p">(</span><span class="bp">self</span>: <span class="kp">&</span><span class="nc">Self</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"my value is {}"</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">value</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>因为我们在 <code>impl MyType</code> 代码块中 <code>Self</code> 就是 <code>MyType</code> 。</p>
<p>很清晰吧?好了,还可以定义其他很多类型作为接收者, <code>Pin<&mut Self></code> 就是其中之一:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">MyType</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">do_thing</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// good luck!1</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>那么 <code>MyType</code> 必须是固定的(pinned)是什么意思呢?比如,它保证不进行转移(move)。除非它实现了 <code>Unpin</code> ,
然后它就可以是非固定的(unpinned),可移动,然后被再一次固定。</p>
<p>对于剩下的文章,我们不会假设我们的 future <code>A</code> 和 <code>B</code> 都是 <code>Unpin</code> ,也就是说我们自己不会移动(move)它们(只销毁(drop)它们)。</p>
<p>你可以说我们不需要 <code>A</code> 和 <code>B</code> 是 <code>Unpin</code> 的,因为我们没有添加指定的 where clause 来标记需要它们是 <code>Unpin</code> 。
因为如果我们需要,我们就要像下面这样添加额外的 <code>trait bound</code> :</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Unpin</span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Unpin</span><span class="p">,</span>
<span class="p">{}</span>
</code></pre></div>
<p>但是我们没有,所以我们不能假设 <code>A</code> 或 <code>B</code> 是 <code>Unpin</code> 的。</p>
<p>所以!我们现在真的只是面临固定(pin)保护的问题。</p>
<p>我们现在只持有一个 <code>Pin<&mut TryJoin<A, B, ...>></code> 但是我希望持续一个 <code>Pin<&mut A></code> (因为这就是我们因为需要轮询 <code>A</code> )。</p>
<p>另外一个解决方法,我倾向于通过一些类似 <a href="https://lib.rs/crates/pin-project">pin-project</a> 包,或者类似 <a href="https://lib.rs/crates/pin-project-lite">pin-project-lite</a>,但是在我们前进的方向直接使用 <code>pin-project</code> 真的很尴尬,
所以我们这里使用 <code>unsafe</code> 作为替代:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">);</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>可以通过编译。但是我们在使用 <code>unsafe</code> ,也就意味着编译器正式停止 ~~照顾~~ 检查我们的代码。
我们自己必须强制执行一些不变量(invariants),并且非常非常小心,同时让其他人审查(review)我们的工作,
但是依然可能会出错,因为他们也会休息。</p>
<p>现在,非常棒的是我们可以轮询 <code>a</code> 。它如果完成会返回 <code>Poll::Ready(Result<AR, E>)</code> ,
否则就是等会会完成则返回 <code>Poll::Pending</code> 。</p>
<p>我们可以观察到:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is pending..."</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is ready!"</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>我们通过打印日志“A is pending”知道准备完成。这可能需要几个回合:毕竟,我们正在做一些重要的事情。
我们建立一个 TCP 连接,接着在上面进行 TLS 会话,接着是一些分开的写,最后读到 EOF(end of file)。</p>
<p>当然,如果我们运行它的话:</p>
<div class="highlight"><pre><span></span><code><span class="n">aytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.495</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.513</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.513</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.513</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.513</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.513</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.514</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.522</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.522</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.522</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.522</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.522</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.523</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.523</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.530</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.530</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">pending</span><span class="o">...</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.530</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">:</span><span class="w"> </span><span class="n">Got</span><span class="w"> </span><span class="n">response</span><span class="o">!</span><span class="w"> </span><span class="n">status</span><span class="o">=</span><span class="n">HTTP</span><span class="o">/</span><span class="mf">1.1</span><span class="w"> </span><span class="mi">200</span><span class="w"> </span><span class="n">OK</span><span class="w"> </span><span class="n">name</span><span class="o">=</span><span class="n">first</span>
<span class="n">Jul</span><span class="w"> </span><span class="mi">25</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">14.530</span><span class="w"> </span><span class="n">INFO</span><span class="w"> </span><span class="n">waytoodeep</span><span class="p">::</span><span class="n">tj</span><span class="p">:</span><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">ready</span><span class="o">!</span>
<span class="n">The</span><span class="w"> </span><span class="n">application</span><span class="w"> </span><span class="n">panicked</span><span class="w"> </span><span class="p">(</span><span class="n">crashed</span><span class="p">)</span><span class="o">.</span>
<span class="n">Message</span><span class="p">:</span><span class="w"> </span><span class="ow">not</span><span class="w"> </span><span class="n">yet</span><span class="w"> </span><span class="n">implemented</span>
<span class="n">Location</span><span class="p">:</span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">tj</span><span class="o">.</span><span class="n">rs</span><span class="p">:</span><span class="mi">46</span>
<span class="n">Backtrace</span><span class="w"> </span><span class="n">omitted</span><span class="o">.</span>
<span class="n">Run</span><span class="w"> </span><span class="n">with</span><span class="w"> </span><span class="n">RUST_BACKTRACE</span><span class="o">=</span><span class="mi">1</span><span class="w"> </span><span class="n">environment</span><span class="w"> </span><span class="n">variable</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">display</span><span class="w"> </span><span class="n">it</span><span class="o">.</span>
<span class="n">Run</span><span class="w"> </span><span class="n">with</span><span class="w"> </span><span class="n">RUST_BACKTRACE</span><span class="o">=</span><span class="n">full</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">include</span><span class="w"> </span><span class="n">source</span><span class="w"> </span><span class="n">snippets</span><span class="o">.</span>
</code></pre></div>
<p>我们可以看到它确实花费了几个回合。</p>
<p>注意如果 <code>A</code> 返回错误我们的代码也会返回 <code>Poll:Ready</code> ,因为我们想收集 A 和 B 的结果。</p>
<p>所以我们对 B 做相同的事情:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is pending..."</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is ready!"</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"B is pending..."</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"B is ready!"</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>然后。。whoops:</p>
<div class="highlight"><pre><span></span><code><span class="n">RUST_LOG</span><span class="o">=</span><span class="n">info</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="o">--</span><span class="n">quiet</span><span class="w"> </span><span class="o">--</span><span class="k">release</span>
<span class="k">error</span><span class="err">[</span><span class="n">E0382</span><span class="err">]</span><span class="o">:</span><span class="w"> </span><span class="k">use</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">moved</span><span class="w"> </span><span class="k">value</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`self`</span>
<span class="w"> </span><span class="o">--></span><span class="w"> </span><span class="n">src</span><span class="o">/</span><span class="n">tj</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">46</span><span class="o">:</span><span class="mi">26</span>
<span class="w"> </span><span class="o">|</span>
<span class="mi">33</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">fn</span><span class="w"> </span><span class="n">poll</span><span class="p">(</span><span class="n">self</span><span class="o">:</span><span class="w"> </span><span class="n">Pin</span><span class="o"><&</span><span class="n">mut</span><span class="w"> </span><span class="n">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="n">mut</span><span class="w"> </span><span class="k">Context</span><span class="o"><</span><span class="s1">'_>) -> Poll<Self::Output> {</span>
<span class="s1"> | ---- move occurs because `self` has type `Pin<&mut TryJoin<A, B, AR, BR, E>>`, which does not implement the `Copy` trait</span>
<span class="s1">34 | let a = unsafe { self.map_unchecked_mut(|this| &mut this.a) };</span>
<span class="s1"> | ------------------------------------- `self` moved due to this method call</span>
<span class="s1">...</span>
<span class="s1">46 | let b = unsafe { self.map_unchecked_mut(|this| &mut this.a) };</span>
<span class="s1"> | ^^^^ value used here after move</span>
<span class="s1"> |</span>
<span class="s1">note: this function takes ownership of the receiver `self`, which moves `self`</span>
<span class="s1"> --> /home/amos/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/pin.rs:776:43</span>
<span class="s1"> |</span>
<span class="s1">776 | pub unsafe fn map_unchecked_mut<U, F>(self, func: F) -> Pin<&'</span><span class="n">a</span><span class="w"> </span><span class="n">mut</span><span class="w"> </span><span class="n">U</span><span class="o">></span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">^^^^</span>
</code></pre></div>
<p>是的。 <code>map_unchecked_mut</code> 占有了 <code>self</code> 。</p>
<p>不用担心,我们可以使用 <code>.as_mut()</code> :</p>
<div class="highlight"><pre><span></span><code><span class="c1">// 👇</span>
<span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">as_mut</span><span class="p">().</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is pending..."</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"A is ready!"</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">as_mut</span><span class="p">().</span><span class="n">map_unchecked_mut</span><span class="p">(</span><span class="o">|</span><span class="n">this</span><span class="o">|</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"B is pending..."</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"B is ready!"</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>但是依然无法通过编译:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
<span class="o">(</span>cut<span class="o">)</span>
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">22</span>:57:07.913<span class="w"> </span>INFO<span class="w"> </span>waytoodeep::tj:<span class="w"> </span>A<span class="w"> </span>is<span class="w"> </span>pending...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">22</span>:57:07.913<span class="w"> </span>INFO<span class="w"> </span>waytoodeep::tj:<span class="w"> </span>A<span class="w"> </span>is<span class="w"> </span>pending...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">22</span>:57:07.913<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">22</span>:57:07.913<span class="w"> </span>INFO<span class="w"> </span>waytoodeep::tj:<span class="w"> </span>A<span class="w"> </span>is<span class="w"> </span>ready!
The<span class="w"> </span>application<span class="w"> </span>panicked<span class="w"> </span><span class="o">(</span>crashed<span class="o">)</span>.
Message:<span class="w"> </span><span class="sb">`</span>async<span class="w"> </span>fn<span class="sb">`</span><span class="w"> </span>resumed<span class="w"> </span>after<span class="w"> </span>completion
Location:<span class="w"> </span>src/main.rs:24
Backtrace<span class="w"> </span>omitted.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>environment<span class="w"> </span>variable<span class="w"> </span>to<span class="w"> </span>display<span class="w"> </span>it.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span>full<span class="w"> </span>to<span class="w"> </span>include<span class="w"> </span><span class="nb">source</span><span class="w"> </span>snippets.
</code></pre></div>
<p>可以看到,一旦 <code>Future</code> 返回 <code>Poll::Ready</code> 我们就不能再次轮询它了。我们为什么会这样?因为 <code>Future</code> 已经返回了结果。如果结果是非 <code>Copy</code> 的,它可能只能返回一次。</p>
<p>所以,我们需要 1)跟踪 <code>A</code> 是否完成,然后 2)在某个地方存储它的返回结果。</p>
<p>我们只需要在我们的结构体中添加一些字段:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">,</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">a_res</span>: <span class="nb">Option</span><span class="o"><</span><span class="n">AR</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b_res</span>: <span class="nb">Option</span><span class="o"><</span><span class="n">BR</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>不要忘记初始化它们:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">TryJoin</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span><span class="p">,</span>
<span class="w"> </span><span class="c1">// 👇</span>
<span class="w"> </span><span class="n">a_res</span>: <span class="nb">None</span><span class="p">,</span>
<span class="w"> </span><span class="n">b_res</span>: <span class="nb">None</span><span class="p">,</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>现在计划是:</p>
<ul>
<li>如果 <code>a_res</code> 是 <code>Some</code> ,然后我们就不需要轮询 <code>a</code> ,因为它已经完成了</li>
<li>同样的逻辑处理 <code>b_res</code> 和 <code>b</code></li>
</ul>
<p>让我们实现它。同时,因为我们已经在使用了 <code>unsafe</code> ,所以我们已经负责维护不变量(invariants),
所以我决定同时固定 <code>a</code> 和 <code>b</code> ,如下:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get_unchecked_mut</span><span class="p">()</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">.</span><span class="n">is_none</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">.</span><span class="n">is_none</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">todo!</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>好了,这个应该能让 <code>a</code> 和 <code>b</code> 有机会在我们崩溃之前完成:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:11:03.851<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:11:04.380<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
The<span class="w"> </span>application<span class="w"> </span>panicked<span class="w"> </span><span class="o">(</span>crashed<span class="o">)</span>.
Message:<span class="w"> </span>not<span class="w"> </span>yet<span class="w"> </span>implemented
Location:<span class="w"> </span>src/tj.rs:69
Backtrace<span class="w"> </span>omitted.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span><span class="m">1</span><span class="w"> </span>environment<span class="w"> </span>variable<span class="w"> </span>to<span class="w"> </span>display<span class="w"> </span>it.
Run<span class="w"> </span>with<span class="w"> </span><span class="nv">RUST_BACKTRACE</span><span class="o">=</span>full<span class="w"> </span>to<span class="w"> </span>include<span class="w"> </span><span class="nb">source</span><span class="w"> </span>snippets.
</code></pre></div>
<p>很好!现在我们需要做的就是解出两个结果并返回它们:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// instead of the `todo!()`:</span>
<span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="nb">Some</span><span class="p">(</span><span class="n">_</span><span class="p">),</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">_</span><span class="p">))</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">.</span><span class="n">take</span><span class="p">().</span><span class="n">unwrap</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">.</span><span class="n">take</span><span class="p">().</span><span class="n">unwrap</span><span class="p">();</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)))</span>
<span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span>
<span class="p">}</span>
</code></pre></div>
<p>可以工作:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:13:32.497<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:13:32.829<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:13:32.829<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
</code></pre></div>
<p>。。。但是这不是 <code>try_join</code> 的实现。我们正在做的和这个完全一样:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// (pseudo-code, buncha things are missing)</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">a</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">b</span><span class="p">.</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">))</span>
<span class="p">}</span>
</code></pre></div>
<p>是顺序的执行的。请记住,仅仅是因为 tokio 的执行器可能用了一堆线程并意味着同时运行是自动的。
前面我们不得不使用 <code>tokio::spwan</code> 或 <code>UnorderedFutures</code> 或 <code>try_join!</code> 让其同时运行。</p>
<p>所以让我们重新看一下。。。当我们轮询 <code>a</code> 的是后发生了什么?</p>
<div class="highlight"><pre><span></span><code><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">.</span><span class="n">is_none</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>额,当轮询 <code>a</code> 的时候返回 <code>Poll::Pending</code> 时我们也会返回 <code>Poll::Pending</code> 。所以这就是问题。
如果 <code>a</code> 正在等待(pending)我们不应该返回。因为如果这时候 <code>b</code> 已经准备好或者有错误呢?</p>
<p>或者如果,我们像这样调用 <code>try_join</code> 呢:</p>
<div class="highlight"><pre><span></span><code><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Joining..."</span><span class="p">);</span>
<span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tj</span>::<span class="n">try_join</span><span class="p">(</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">sleep</span><span class="p">(</span><span class="n">Duration</span>::<span class="n">from_millis</span><span class="p">(</span><span class="mi">2000</span><span class="p">)).</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="k">async</span><span class="w"> </span><span class="k">move</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">sleep</span><span class="p">(</span><span class="n">Duration</span>::<span class="n">from_millis</span><span class="p">(</span><span class="mi">10</span><span class="p">)).</span><span class="k">await</span><span class="p">;</span>
<span class="w"> </span><span class="nb">Err</span>::<span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="p">(</span><span class="n">color_eyre</span>::<span class="n">eyre</span>::<span class="n">eyre</span><span class="o">!</span><span class="p">(</span><span class="s">"uh oh"</span><span class="p">))</span>
<span class="w"> </span><span class="p">},</span>
<span class="p">)</span>
<span class="p">.</span><span class="k">await</span><span class="p">;</span>
</code></pre></div>
<p>。。。然后 <code>a</code> 花费了 2 秒钟才准备好,同时 <code>b</code> 会在 10毫秒之后返回一个错误,如果我们轮询它!</p>
<p>嗐,我们并没有:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:19:26.972<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:19:28.990<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=</span>Err<span class="o">(</span>
<span class="w"> </span><span class="m">0</span>:<span class="w"> </span>uh<span class="w"> </span>oh
Location:
<span class="w"> </span>src/main.rs:28
<span class="o">(</span>cut<span class="o">)</span>
</code></pre></div>
<p>(注意时间戳)</p>
<p>重点是 <code>try_join</code> 会提前失败:一旦 Future 返回 <code>Result::Err~</code> 。</p>
<p>所以我们必须同时轮询 <code>a</code> 和 <code>b</code> 。好吧。。。不是严格意义的同时。我们必须每次我们的 <code>TryJoin</code> future 对象被轮询时并发(concurrently)的轮询它们,
直到它们返回结果。</p>
<p>有一个简单解决办法 -- 在任一 future 对象返回 <code>Poll::Pending</code> 时不返回 <code>Poll::Pending</code> 。</p>
<p>同时,我厌倦了输入 <code>Poll::Ready</code> 并且 <code>Poll<T></code> 实现了 <code>From<T></code> ,所以我们可以使用 <code>.into()</code> 了:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get_unchecked_mut</span><span class="p">()</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">.</span><span class="n">is_none</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">.</span><span class="n">is_none</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="nb">Some</span><span class="p">(</span><span class="n">_</span><span class="p">),</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="n">_</span><span class="p">))</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a_res</span><span class="p">.</span><span class="n">take</span><span class="p">().</span><span class="n">unwrap</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b_res</span><span class="p">.</span><span class="n">take</span><span class="p">().</span><span class="n">unwrap</span><span class="p">();</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)).</span><span class="n">into</span><span class="p">()</span>
<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>让我们再次运行</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:22:40.238<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:22:40.253<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=</span>Err<span class="o">(</span>
<span class="w"> </span><span class="m">0</span>:<span class="w"> </span>uh<span class="w"> </span>oh
Location:
<span class="w"> </span>src/main.rs:28
<span class="o">(</span>cut<span class="o">)</span>
</code></pre></div>
<p>。。。可以了!我是说它按照预期的失败了。预期的失败就是成功。</p>
<p>然后我们将这个方法应用到调用 <code>try_join</code> :</p>
<div class="highlight"><pre><span></span><code><span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Joining..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tj</span>::<span class="n">try_join</span><span class="p">(</span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"first"</span><span class="p">),</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"second"</span><span class="p">)).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">?</span><span class="n">res</span><span class="p">,</span><span class="w"> </span><span class="s">"All done!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>我们可以看到竞争又回来了:有时 <code>first</code> 先完成,有时则不是:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:25.925<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:26.224<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:26.236<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:26.236<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:26.937<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:27.237<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:27.242<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:27.242<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:27.865<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:28.164<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:28.818<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:28.818<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:30.153<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:31.477<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:31.496<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:25:31.496<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
</code></pre></div>
<p>。。。同时结果的顺序是正确的。</p>
<p>非常好,我们实现了它!</p>
<p>但是!</p>
<h3 id="我们可以做的更好">我们可以做的更好</h3>
<p>幸运的是,坏就是好。</p>
<p>下面这个类型困扰着我:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">,</span>
<span class="w"> </span><span class="n">a_res</span>: <span class="nb">Option</span><span class="o"><</span><span class="n">AR</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b_res</span>: <span class="nb">Option</span><span class="o"><</span><span class="n">BR</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>我其实只有在 <code>a</code> 完成后才需要 <code>a_res</code> 。一旦 <code>a</code> 完成然后将结果存储到 <code>a_res</code> ,我们就不再需要 <code>a</code> 了。</p>
<p>实际上,甚至我们不应该再碰 <code>a</code> 。</p>
<p>这听起来更像我们要么持有 <code>A</code> 要么持有 <code>AR</code> ,但是永远不会同时持有。</p>
<blockquote>
<p><code>SUM TYPES</code> : Rust 的枚举就是一个汇总类型。</p>
</blockquote>
<p>所以!汇总类型。Rust 枚举。这就是我们想要的。让我们创建一个叫做 <code>State</code> 的类型,然后它有两个变体:
一个用于它还是 future 对象,一个用于它是一个结果。非常简单!</p>
<div class="highlight"><pre><span></span><code><span class="k">enum</span> <span class="nc">State</span><span class="o"><</span><span class="n">F</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">F</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">Future</span><span class="p">(</span><span class="n">F</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">T</span><span class="p">),</span>
<span class="p">}</span>
</code></pre></div>
<p>这将会非常棒!</p>
<p>让我们赋给我们的 <code>TryJoin</code> 结构体:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span><span class="o"><</span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>(是不是非常漂亮)</p>
<p>然后初始化它们:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">TryJoin</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>非常酷。然后我们只需要稍微调整一下我们的 <code>poll</code> 方法,我们需要将 <code>Pin<&mut Self></code> 转换为 <code>&mut Self</code> 。。。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get_unchecked_mut</span><span class="p">()</span><span class="w"> </span><span class="p">};</span>
</code></pre></div>
<p>这是可以的,因为我们承诺维护不变量,也就是说我们不会转移(move) <code>State::Future</code> 内部。</p>
<p>然后如果 <code>a</code> 是 <code>State::Future</code> 我们就轮询它,然后我们再传播错误或者保存它的结果:</p>
<div class="highlight"><pre><span></span><code><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="n">a</span><span class="p">)</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">t</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">t</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nb">Err</span><span class="p">(</span><span class="n">e</span><span class="p">).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>然后同样的修改 <code>b</code> 。。。</p>
<div class="highlight"><pre><span></span><code><span class="c1">// you can figure that one out, I believe in you</span>
</code></pre></div>
<p>然后我们就完成了如果它们都是 <code>State::Ok</code> !否则我们就返回 <code>Poll::Pending</code> :</p>
<div class="highlight"><pre><span></span><code><span class="k">match</span><span class="w"> </span><span class="p">(</span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">a</span><span class="p">),</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">b</span><span class="p">))</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>非常好。</p>
<p>除了它无法通过编译:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
error<span class="o">[</span>E0507<span class="o">]</span>:<span class="w"> </span>cannot<span class="w"> </span>move<span class="w"> </span>out<span class="w"> </span>of<span class="w"> </span><span class="sb">`</span>this.a<span class="sb">`</span><span class="w"> </span>which<span class="w"> </span>is<span class="w"> </span>behind<span class="w"> </span>a<span class="w"> </span>mutable<span class="w"> </span>reference
<span class="w"> </span>--><span class="w"> </span>src/tj.rs:65:16
<span class="w"> </span><span class="p">|</span>
<span class="m">65</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>match<span class="w"> </span><span class="o">(</span>this.a,<span class="w"> </span>this.b<span class="o">)</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="p">|</span><span class="w"> </span>^^^^^^<span class="w"> </span>move<span class="w"> </span>occurs<span class="w"> </span>because<span class="w"> </span><span class="sb">`</span>this.a<span class="sb">`</span><span class="w"> </span>has<span class="w"> </span><span class="nb">type</span><span class="w"> </span><span class="sb">`</span>State<A,<span class="w"> </span>AR,<span class="w"> </span>E><span class="sb">`</span>,<span class="w"> </span>which<span class="w"> </span>does<span class="w"> </span>not<span class="w"> </span>implement<span class="w"> </span>the<span class="w"> </span><span class="sb">`</span>Copy<span class="sb">`</span><span class="w"> </span>trait
error<span class="o">[</span>E0507<span class="o">]</span>:<span class="w"> </span>cannot<span class="w"> </span>move<span class="w"> </span>out<span class="w"> </span>of<span class="w"> </span><span class="sb">`</span>this.b<span class="sb">`</span><span class="w"> </span>which<span class="w"> </span>is<span class="w"> </span>behind<span class="w"> </span>a<span class="w"> </span>mutable<span class="w"> </span>reference
<span class="w"> </span>--><span class="w"> </span>src/tj.rs:65:24
<span class="w"> </span><span class="p">|</span>
<span class="m">65</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>match<span class="w"> </span><span class="o">(</span>this.a,<span class="w"> </span>this.b<span class="o">)</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="p">|</span><span class="w"> </span>^^^^^^<span class="w"> </span>move<span class="w"> </span>occurs<span class="w"> </span>because<span class="w"> </span><span class="sb">`</span>this.b<span class="sb">`</span><span class="w"> </span>has<span class="w"> </span><span class="nb">type</span><span class="w"> </span><span class="sb">`</span>State<B,<span class="w"> </span>BR,<span class="w"> </span>E><span class="sb">`</span>,<span class="w"> </span>which<span class="w"> </span>does<span class="w"> </span>not<span class="w"> </span>implement<span class="w"> </span>the<span class="w"> </span><span class="sb">`</span>Copy<span class="sb">`</span><span class="w"> </span>trait
error:<span class="w"> </span>aborting<span class="w"> </span>due<span class="w"> </span>to<span class="w"> </span><span class="m">2</span><span class="w"> </span>previous<span class="w"> </span>errors
For<span class="w"> </span>more<span class="w"> </span>information<span class="w"> </span>about<span class="w"> </span>this<span class="w"> </span>error,<span class="w"> </span>try<span class="w"> </span><span class="sb">`</span>rustc<span class="w"> </span>--explain<span class="w"> </span>E0507<span class="sb">`</span>.
error:<span class="w"> </span>could<span class="w"> </span>not<span class="w"> </span>compile<span class="w"> </span><span class="sb">`</span>waytoodeep<span class="sb">`</span>
To<span class="w"> </span>learn<span class="w"> </span>more,<span class="w"> </span>run<span class="w"> </span>the<span class="w"> </span><span class="nb">command</span><span class="w"> </span>again<span class="w"> </span>with<span class="w"> </span>--verbose.
</code></pre></div>
<p>因为。。。我们只有 <code>&mut Self</code> 而不是 <code>Self</code> 。</p>
<p>我们没有自己的所有权,仅仅是借用我们自己。</p>
<p>所以,我们不能将将我们的成员转移(move)出去,因为我们不能阻止其他人再次轮询 <code>TryJoin</code> 。
所以这种情况,我们需要崩溃(panic)。</p>
<p>当然,如果我们像 <code>Option<T></code> 那样有一个 <code>.take()</code> 方法事情就会大大简化。
它返回 Option 拥有的任何内容,或者 <code>None</code> 。</p>
<p>但是我们没有 <code>None</code> 。我们只有 <code>State::Future</code> 和 <code>State::OK</code> ,没有“中立”(neutral)状态。</p>
<p>让我们创建一个:</p>
<div class="highlight"><pre><span></span><code><span class="k">enum</span> <span class="nc">State</span><span class="o"><</span><span class="n">F</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">F</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">Future</span><span class="p">(</span><span class="n">F</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">T</span><span class="p">),</span>
<span class="w"> </span><span class="n">Gone</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>现在,我们可以将 <code>this.a</code> 和 <code>this.b</code> 替换为 <code>State::Gone</code> 。。。或者它的返回结果(我们拥有所有权)。
然后我们就可以将它们转移(move)出去。</p>
<p>但是同时。。。我们需要再次对其进行模式匹配(pattern match)。</p>
<p>就像:</p>
<div class="highlight"><pre><span></span><code><span class="k">match</span><span class="w"> </span><span class="p">(</span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">this</span><span class="p">.</span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">),</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">))</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">std</span>::<span class="n">mem</span>::<span class="n">replace</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">State</span>::<span class="n">Gone</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">t</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">t</span><span class="p">,</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">unreachable!</span><span class="p">(),</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">std</span>::<span class="n">mem</span>::<span class="n">replace</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">this</span><span class="p">.</span><span class="n">b</span><span class="p">,</span><span class="w"> </span><span class="n">State</span>::<span class="n">Gone</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">t</span><span class="p">)</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">t</span><span class="p">,</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">unreachable!</span><span class="p">(),</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)).</span><span class="n">into</span><span class="p">()</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>实话说。。。我看过更糟的代码。它只是没那么<a href="https://en.wikipedia.org/wiki/Don%27t%5Frepeat%5Fyourself">DRY</a>。</p>
<p>非常好的实现!</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:52:24.097<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:52:25.050<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:52:25.061<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">23</span>:52:25.061<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
</code></pre></div>
<p>看,只有 11ms 的间隔。</p>
<h3 id="更进一步">更进一步?</h3>
<p>这段代码再次困扰了我:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span><span class="o"><</span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>因为现在 <code>a</code> 和 <code>b</code> 是三态的(tri-state): <code>Future</code> 、 <code>Ok</code> 或者 <code>Gone</code> 。</p>
<p>如果 <code>a</code> 和 <code>b</code> 都是 <code>Gone</code> 呢?这个状态不合理!</p>
<p>如果发生了这个状态,我们现在将会永远返回 <code>Poll::Pending</code> -- 这不太好 -- 一个死锁。</p>
<p>我们真正想要的是。。。两个枚举。实际上我们想要整个 <code>TryJoin</code> 类型是一个 <code>enum</code> 。</p>
<div class="highlight"><pre><span></span><code><span class="k">enum</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span><span class="o"><</span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="n">Done</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<p>像这样初始化:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">TryJoin</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>然后,surprice! <code>Poll<T></code> 实现了 <a href="https://doc.rust-lang.org/stable/std/ops/trait.Try.html">Try</a> trait。所以我们可以使用 <code>?</code> 。
所以最终我们的代码实际上非常短小:</p>
<div class="highlight"><pre><span></span><code><span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get_unchecked_mut</span><span class="p">()</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Done</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">panic!</span><span class="p">(</span><span class="s">"TryJoin future polled after completion"</span><span class="p">),</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="p">}.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="o">*</span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">res</span><span class="o">?</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="p">}.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="o">*</span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">res</span><span class="o">?</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">),</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">))</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">std</span>::<span class="n">mem</span>::<span class="n">replace</span><span class="p">(</span><span class="n">this</span><span class="p">,</span><span class="w"> </span><span class="bp">Self</span>::<span class="n">Done</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">unreachable!</span><span class="p">(),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>现在,我知道你在想什么。 <code>Pin<&mut T></code> 不是恰恰用来避免像 <code>std::mem::swap</code> 和 <code>std::mem::replace</code> 吗?
这些所有的转移(move)都是围绕着内存!这是被禁止的!是的,我们承诺了不去转移(move)它。
但是在这个情况下,我们只是在完成轮询两个 future 对象后转移了 <code>self</code> / <code>this</code> 。</p>
<p>然后我们就再也没有使用过两个 future 对象,无论固定还是非固定。同时我们从来也没保证过结果自身是否将要被固定(pinned)!</p>
<p>我们只需要决定某些东西是“永远固定”还是“永不固定”,然后我们可能会编写结果正确的代码。</p>
<p>在我们的场景下,只有 <code>TryJoin::Polling(State::Future(_))</code> 就是“永远固定” 的,其他都不是。</p>
<p>当然,我们快速的从 <code>Pin<&mut Self></code> 切换到 <code>&mut Self</code> ,然后又回到 <code>Pin<&mut A></code> ,
但只要我们不要在中间移动就没有问题。</p>
<p>如果我们在持有 future 对象的情况下使用 <code>std::mem:replace</code> 或 <code>std::mem::swap</code> 就会不妙。
所以,我们还好,我想,我不太确定。如果不是,有人应该会留言。</p>
<h3 id="就这样">就这样</h3>
<p>让我们回顾我们的工作:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/tj.rs`</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="p">{</span>
<span class="w"> </span><span class="n">future</span>::<span class="n">Future</span><span class="p">,</span>
<span class="w"> </span><span class="n">pin</span>::<span class="n">Pin</span><span class="p">,</span>
<span class="w"> </span><span class="n">task</span>::<span class="p">{</span><span class="n">Context</span><span class="p">,</span><span class="w"> </span><span class="n">Poll</span><span class="p">},</span>
<span class="p">};</span>
<span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">try_join</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="n">a</span>: <span class="nc">A</span><span class="p">,</span><span class="w"> </span><span class="n">b</span>: <span class="nc">B</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">impl</span><span class="w"> </span><span class="n">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">>></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">TryJoin</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
<span class="k">enum</span> <span class="nc">State</span><span class="o"><</span><span class="n">F</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">F</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">Future</span><span class="p">(</span><span class="n">F</span><span class="p">),</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">T</span><span class="p">),</span>
<span class="p">}</span>
<span class="k">enum</span> <span class="nc">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span><span class="o"><</span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">,</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="n">Done</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="w"> </span><span class="n">Future</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">TryJoin</span><span class="o"><</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">></span>
<span class="k">where</span>
<span class="w"> </span><span class="n">A</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="w"> </span><span class="n">B</span>: <span class="nc">Future</span><span class="o"><</span><span class="n">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="n">BR</span><span class="p">,</span><span class="w"> </span><span class="n">E</span><span class="o">>></span><span class="p">,</span>
<span class="p">{</span>
<span class="w"> </span><span class="k">type</span> <span class="nc">Output</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Result</span><span class="o"><</span><span class="p">(</span><span class="n">AR</span><span class="p">,</span><span class="w"> </span><span class="n">BR</span><span class="p">),</span><span class="w"> </span><span class="n">E</span><span class="o">></span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="bp">self</span>: <span class="nc">Pin</span><span class="o"><&</span><span class="k">mut</span><span class="w"> </span><span class="bp">Self</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">cx</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Context</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span><span class="p">)</span><span class="w"> </span>-> <span class="nc">Poll</span><span class="o"><</span><span class="bp">Self</span>::<span class="n">Output</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get_unchecked_mut</span><span class="p">()</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Done</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">panic!</span><span class="p">(</span><span class="s">"TryJoin future polled after completion"</span><span class="p">),</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="p">}.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="o">*</span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">res</span><span class="o">?</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">State</span>::<span class="n">Future</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Ready</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">Pin</span>::<span class="n">new_unchecked</span><span class="p">(</span><span class="n">fut</span><span class="p">)</span><span class="w"> </span><span class="p">}.</span><span class="n">poll</span><span class="p">(</span><span class="n">cx</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="o">*</span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">res</span><span class="o">?</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="p">(</span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">),</span><span class="w"> </span><span class="n">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">_</span><span class="p">))</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="k">match</span><span class="w"> </span><span class="n">std</span>::<span class="n">mem</span>::<span class="n">replace</span><span class="p">(</span><span class="n">this</span><span class="p">,</span><span class="w"> </span><span class="bp">Self</span>::<span class="n">Done</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="bp">Self</span>::<span class="n">Polling</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">a</span>: <span class="nc">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">a</span><span class="p">),</span>
<span class="w"> </span><span class="n">b</span>: <span class="nc">State</span>::<span class="nb">Ok</span><span class="p">(</span><span class="n">b</span><span class="p">),</span>
<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="nb">Ok</span><span class="p">((</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">)).</span><span class="n">into</span><span class="p">(),</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="fm">unreachable!</span><span class="p">(),</span>
<span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="n">_</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">Poll</span>::<span class="n">Pending</span><span class="p">,</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>还有我们小小的 HTTPS 客户端:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// in `src/main.rs`</span>
<span class="k">use</span><span class="w"> </span><span class="n">color_eyre</span>::<span class="n">Report</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="p">{</span><span class="n">net</span>::<span class="n">SocketAddr</span><span class="p">,</span><span class="w"> </span><span class="n">sync</span>::<span class="n">Arc</span><span class="p">};</span>
<span class="k">use</span><span class="w"> </span><span class="n">tokio</span>::<span class="p">{</span>
<span class="w"> </span><span class="n">io</span>::<span class="p">{</span><span class="n">AsyncReadExt</span><span class="p">,</span><span class="w"> </span><span class="n">AsyncWriteExt</span><span class="p">},</span>
<span class="w"> </span><span class="n">net</span>::<span class="n">TcpStream</span><span class="p">,</span>
<span class="p">};</span>
<span class="k">use</span><span class="w"> </span><span class="n">tokio_rustls</span>::<span class="p">{</span><span class="n">rustls</span>::<span class="n">ClientConfig</span><span class="p">,</span><span class="w"> </span><span class="n">TlsConnector</span><span class="p">};</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing</span>::<span class="n">info</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">EnvFilter</span><span class="p">;</span>
<span class="k">use</span><span class="w"> </span><span class="n">webpki</span>::<span class="n">DNSNameRef</span><span class="p">;</span>
<span class="k">mod</span> <span class="nn">tj</span><span class="p">;</span>
<span class="cp">#[tokio::main(flavor = </span><span class="s">"current_thread"</span><span class="cp">)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">setup</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="s">"Joining..."</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">res</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tj</span>::<span class="n">try_join</span><span class="p">(</span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"first"</span><span class="p">),</span><span class="w"> </span><span class="n">fetch_thing</span><span class="p">(</span><span class="s">"second"</span><span class="p">)).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">?</span><span class="n">res</span><span class="p">,</span><span class="w"> </span><span class="s">"All done!"</span><span class="p">);</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="cp">#[allow(dead_code)]</span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fetch_thing</span><span class="p">(</span><span class="n">name</span>: <span class="kp">&</span><span class="kt">str</span><span class="p">)</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><&</span><span class="kt">str</span><span class="p">,</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// look out it's port 443 now</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">addr</span>: <span class="nc">SocketAddr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">],</span><span class="w"> </span><span class="mi">443</span><span class="p">).</span><span class="n">into</span><span class="p">();</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">socket</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">TcpStream</span>::<span class="n">connect</span><span class="p">(</span><span class="n">addr</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// establish a TLS session...</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">connector</span>: <span class="nc">TlsConnector</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">config</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ClientConfig</span>::<span class="n">new</span><span class="p">();</span>
<span class="w"> </span><span class="n">config</span>
<span class="w"> </span><span class="p">.</span><span class="n">root_store</span>
<span class="w"> </span><span class="p">.</span><span class="n">add_server_trust_anchors</span><span class="p">(</span><span class="o">&</span><span class="n">webpki_roots</span>::<span class="n">TLS_SERVER_ROOTS</span><span class="p">);</span>
<span class="w"> </span><span class="n">Arc</span>::<span class="n">new</span><span class="p">(</span><span class="n">config</span><span class="p">).</span><span class="n">into</span><span class="p">()</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">dnsname</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DNSNameRef</span>::<span class="n">try_from_ascii_str</span><span class="p">(</span><span class="s">"one.one.one.one"</span><span class="p">)</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">socket</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">connector</span><span class="p">.</span><span class="n">connect</span><span class="p">(</span><span class="n">dnsname</span><span class="p">,</span><span class="w"> </span><span class="n">socket</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// we're writing straight to the socket, there's no buffering</span>
<span class="w"> </span><span class="c1">// so no need to flush</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"GET / HTTP/1.1</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Host: one.one.one.one</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"User-Agent: cool-bear</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"Connection: close</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">write_all</span><span class="p">(</span><span class="s">b"</span><span class="se">\r\n</span><span class="s">"</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">String</span>::<span class="n">with_capacity</span><span class="p">(</span><span class="mi">256</span><span class="p">);</span>
<span class="w"> </span><span class="n">socket</span><span class="p">.</span><span class="n">read_to_string</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">response</span><span class="p">).</span><span class="k">await</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">status</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">response</span><span class="p">.</span><span class="n">lines</span><span class="p">().</span><span class="n">next</span><span class="p">().</span><span class="n">unwrap_or_default</span><span class="p">();</span>
<span class="w"> </span><span class="n">info</span><span class="o">!</span><span class="p">(</span><span class="o">%</span><span class="n">status</span><span class="p">,</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="s">"Got response!"</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// dropping the socket will close the connection</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">setup</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Result</span><span class="o"><</span><span class="p">(),</span><span class="w"> </span><span class="n">Report</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LIB_BACKTRACE"</span><span class="p">,</span><span class="w"> </span><span class="s">"1"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">color_eyre</span>::<span class="n">install</span><span class="p">()</span><span class="o">?</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">).</span><span class="n">is_err</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">std</span>::<span class="n">env</span>::<span class="n">set_var</span><span class="p">(</span><span class="s">"RUST_LOG"</span><span class="p">,</span><span class="w"> </span><span class="s">"info"</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">tracing_subscriber</span>::<span class="n">fmt</span>::<span class="n">fmt</span><span class="p">()</span>
<span class="w"> </span><span class="p">.</span><span class="n">with_env_filter</span><span class="p">(</span><span class="n">EnvFilter</span>::<span class="n">from_default_env</span><span class="p">())</span>
<span class="w"> </span><span class="p">.</span><span class="n">init</span><span class="p">();</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(())</span>
<span class="p">}</span>
</code></pre></div>
<p>And it works.</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>cargo<span class="w"> </span>run<span class="w"> </span>--quiet<span class="w"> </span>--release
Jul<span class="w"> </span><span class="m">26</span><span class="w"> </span><span class="m">00</span>:08:13.399<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Joining...
Jul<span class="w"> </span><span class="m">26</span><span class="w"> </span><span class="m">00</span>:08:13.707<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>first
Jul<span class="w"> </span><span class="m">26</span><span class="w"> </span><span class="m">00</span>:08:13.709<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>Got<span class="w"> </span>response!<span class="w"> </span><span class="nv">status</span><span class="o">=</span>HTTP/1.1<span class="w"> </span><span class="m">200</span><span class="w"> </span>OK<span class="w"> </span><span class="nv">name</span><span class="o">=</span>second
Jul<span class="w"> </span><span class="m">26</span><span class="w"> </span><span class="m">00</span>:08:13.710<span class="w"> </span>INFO<span class="w"> </span>waytoodeep:<span class="w"> </span>All<span class="w"> </span><span class="k">done</span>!<span class="w"> </span><span class="nv">res</span><span class="o">=(</span><span class="s2">"first"</span>,<span class="w"> </span><span class="s2">"second"</span><span class="o">)</span>
</code></pre></div>
<p>2ms 间隔!这是一个新的记录。</p>从零开始用 Emacs 管理笔记2021-07-28T00:00:00+08:002021-07-28T00:00:00+08:00coldtag:www.linuxzen.com,2021-07-28:/dump-brain-with-emacs.html<h1 id="_1">用过的笔记管理软件</h1>
<p>我探索过很多笔记管理软件,从最开始用 Vim 的时候使用 Vimwiki,后面转向自托管的 leanote。后面编辑器切换到 …</p><h1 id="_1">用过的笔记管理软件</h1>
<p>我探索过很多笔记管理软件,从最开始用 Vim 的时候使用 Vimwiki,后面转向自托管的 leanote。后面编辑器切换到 Emacs 尝试过 org-wiki,
最终现在开始使用 org-roam。org-roam 基于 Emacs + Orgmode 的 <a href="https://roamresearch.com/">Roam Research</a> 开源版本。</p>
<h1 id="_2">具体配置过程</h1>
<p>参见 <a href="https://coldnight.github.io/dump-brain-with-emacs/posts/dump-brain-with-emacs/">从零开始用 Emacs 管理笔记</a>。</p>【译】Python 幕后 #1: CPython 虚拟机如何工作2020-09-08T00:00:00+08:002020-09-08T00:00:00+08:00coldtag:www.linuxzen.com,2020-09-08:/python-behind-the-scenes_1_how_cpython_vm_works.html<p>原文链接:<a href="https://tenthousandmeters.com/blog/python-behind-the-scenes-1-how-the-cpython-vm-works/">Python behind the scenes #1: how the CPython VM works</a>。</p>
<h3 id="介绍-introduction">介绍(Introduction)</h3>
<p>你是否曾经好奇过当你运行 Python 代码时 <code>python</code> 做了些什么?</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>python<span class="w"> </span>script.py
</code></pre></div>
<p>这篇文章将开启一 …</p><p>原文链接:<a href="https://tenthousandmeters.com/blog/python-behind-the-scenes-1-how-the-cpython-vm-works/">Python behind the scenes #1: how the CPython VM works</a>。</p>
<h3 id="介绍-introduction">介绍(Introduction)</h3>
<p>你是否曾经好奇过当你运行 Python 代码时 <code>python</code> 做了些什么?</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>python<span class="w"> </span>script.py
</code></pre></div>
<p>这篇文章将开启一个系列来尝试解答这个问题。我们将深入 Python 最流行的实现 CPython 的内部。
通过深入 CPython 的内部我们将更深一层的去理解这门编程语言本身。这也是我们这个系列的最主要的目标。
如果你熟悉 Python 并且可以阅读 C 代码,但是对 CPython 源码本身没有太多的经验,
那么你可能非常适合本系列,并且对本系列感兴趣。</p>
<h3 id="什么是-cpython-并且为什么有人想学习它-what-cpython-is-and-why-anyone-would-want-to-study-it">什么是 CPython 并且为什么有人想学习它(What CPython is and why anyone would want to study it)</h3>
<p>我们首先来说明一些众所周知的事情。CPython 是用 C 编写的 Python 解析器。他是 Python 语言的众多实现
的一种,其他还有诸如 PyPy、Jython、IronPython 等。CPython 的独特之处在于它是 Python 的起源、维护时间最长也是最受欢迎的。</p>
<p>CPython 实现了 Python,但是 Python 是什么?最简单的一个答案可能是:Python 是一门编程语言。
当正确问相同的问题,那么答案将会更加明确:什么定义了 Python?Python 不像 C 语言有正式的规范,
但是与之相近的是 <a href="https://docs.python.org/zh-cn/3.9/reference/index.html">Python 语言参考(Python Language Reference)</a>,它以如下内容开始:</p>
<blockquote>
<p>我希望尽可能地保证内容精确无误,但还是选择使用自然词句进行描述,正式的规格定义仅用于句法和词法解析。这样应该能使文档对于普通人来说更易理解,但也可能导致一些歧义。因此,如果你是来自火星并且想凭借这份文档把 Python 重新实现一遍,也许有时需要自行猜测,实际上最终大概会得到一个十分不同的语言。而在另一方面,如果你正在使用 Python 并且想了解有关该语言特定领域的精确规则,你应该能够在这里找到它们。如果你希望查看对该语言更正式的定义,也许你可以花些时间自己写上一份 --- 或者发明一台克隆机器 :-)</p>
</blockquote>
<p>所以 Python 不仅仅通过语言参考定义,说 Python 是通过语言参考定义的实现或者说是 CPython 都是错误的,
因为其中的一些实现细节并不是语言的一部分。一个基于引用计数的垃圾回收器就是例子。由于没有一个来说法是正确的,
我们可以说 Python 的一部分是由 Python 语言参考(Python Language Reference)定义,
一部分是它的主要实现 CPython 定义。</p>
<p>这样的结论似乎很古怪,但是我认为这对我们弄清我们要学习的主题至关重要。我们可能依然困惑我们为什么需要学习它。
除了好奇心,我认为还有如下理由:</p>
<ul>
<li>拥有完整的视角可以更深入的理解这门语言。如果了解一些 Python 的细节那么就更容易掌握一些 Python 特性。</li>
<li>在实践中实现细节很重要。当想要了解语言适用性及其局限性、评估性能或检测效率低下时,了解对象如何存储,
垃圾回收器如何工作,以及如何协调多个线程将是非常重要的。</li>
<li>CPython 提供了 Python/C API 来允许我们用 C 扩展 Python 或者在 C 中嵌入 Python。
程序员需要很好的理解 CPython 如何工作才能高效的使用这些 API。</li>
</ul>
<h3 id="了解-cpython-如果工作需要做些什么-what-it-takes-to-understand-how-cpython-works">了解 CPython 如果工作需要做些什么(What it takes to understand how CPython works)</h3>
<p>CPython 被设计成易于维护。一个新人完全可以阅读源代码并理解代码做了些什么。但是,这可能需要一些时间。
通过这个系列我希望能帮助你缩短这个时间。</p>
<h3 id="这个系列如何推进-how-this-series-is-laid-out">这个系列如何推进(How this series is laid out)</h3>
<p>我选择采取自上而下的方法。在这个部分我们将探索 CPython 虚拟机的核心概念。接下来,我们将看到
CPython 如何编译一个程序到 VM 可以执行的内容。再然后,我们将熟悉源代码,并通过执行一个程序
来学习解释器的主要部分。最后,我们可以挑选语言不同的方面来一个接一个的去看看它们是如何实现的。
这是我的一个大概的想法,并不是一个严格的计划。</p>
<p><strong><em>*Note</em></strong>*: 本文参考 CPython 3.9。一些实现细节将必然会随着 CPython 的演进而改变。
我将会尝试关注一些重要的改变并添加更新备注。</p>
<h3 id="鸟瞰-the-big-picture">鸟瞰(The big picture)</h3>
<p>执行一个 Python 程序大概经过三个阶段:</p>
<ol>
<li>初始化(Initialization)</li>
<li>编译(Compilation)</li>
<li>解释(Interpretation)</li>
</ol>
<p>在初始化阶段,CPython 初始化运行 Python 所需要的数据结构。同时也准备一些诸如
内建类型、配置和加载内建模块,初始化导入系统(import system)和一些其他的事情。
这是一个非常重要的阶段,但是由于其功能性质这个阶段也是常被 CPython 的探索者忽略的一个阶段。</p>
<p>接下来是编译阶段。CPython 在某种意义上是一个解释器而不是编译器,因为它不输出机器码。
但是解释器通常会在执行之前把源代码翻译成一种中间语言(intermediate representation)。
CPython 也是如此。这个翻译阶段和一个典型的编译器做同样的事情:解析源代码然后构建 AST(Abstract Syntax Tree)、
通过 AST 生成字节码、甚至执行一些字节码优化的操作。</p>
<p>在进入下一阶段之前,我们需要理解什么是字节码(bytecode)。字节码是一系列的指令。
每一个指令由两个字节组成:一个为 opcode,一个为参数(argument)。看如下例子:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">3</span>
</code></pre></div>
<p>CPython 将函数定义 <code>g</code> 翻译成一个字节序列:[124, 0, 100, 1, 23, 0, 83, 0]。
如果我们运行标准库 <code>dis</code> <sup id="fnref:fn:1"><a class="footnote-ref" href="#fn:fn:1">1</a></sup>去反汇编它,我们将会得到如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="mi">2</span> <span class="mi">0</span> <span class="n">LOAD_FAST</span> <span class="mi">0</span> <span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="mi">2</span> <span class="n">LOAD_CONST</span> <span class="mi">1</span> <span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="mi">4</span> <span class="n">BINARY_ADD</span>
<span class="mi">6</span> <span class="n">RETURN_VALUE</span>
</code></pre></div>
<p>字节 <code>124</code> 表示 opcode <code>LOAD_FAST</code> 并且有一个参数 <code>0</code> 。字节 <code>100</code> 表示 opcode <code>LOAD_CONST</code> 并且有一个参数 <code>1</code> 。
<code>BINARY_ADD</code> 和 <code>RETURN_VALUE</code> 指令不需要参数所以总是被编码成 <code>(23, 0)</code> 和 <code>(83, 0)</code> 。</p>
<p>CPython 的核心就是一个运行字节码的虚拟机。通过查看上面例子你可能已经猜到它是如何工作的拉。
CPython 虚拟机一个基于栈的。也就意味这它执行指令并通过栈存储和获取数据。
<code>LOAD_FAST</code> 指令将局部变量压入栈,
<code>LOAD_CONST</code> 压入一个常量,
<code>BINARY_ADD</code> 从栈中弹出两个对象,然后进行相加并将结果放回栈。
最好 <code>RETURN_VALUE</code> 从栈弹出任意值然后将结果返回给调用者。</p>
<p>当有指令需要运行时字节码运行在一个巨大的执行循环中, <code>yield</code> 一个值或者发生错误将导致它停止。</p>
<p>这样一个简短概述引发了很多问题:</p>
<ul>
<li>参数对 opcode <code>LOAD_FAST</code> 和 <code>LOAD_CONST</code> 的意义是什么?他们是索引吗?他们如何索引?</li>
<li>VM 会在栈上放置值或者对象的引用吗?</li>
<li>CPython 如何知道 <code>x</code> 是一个局部变量。</li>
<li>如果参数太大无法放到一个字节内怎么办?</li>
<li>连接两个字符串和两个数字相加是同一个指令吗?如果是,VM 如何处理这些操作之间的差异?</li>
</ul>
<p>要回答这些和其他一些有趣的问题我们需要先看一下 CPython VM 的一些核心概念。</p>
<h3 id="代码对象-函数对象-帧-code-objects-function-objects-frames">代码对象、函数对象、帧(Code objects, function objects, frames)</h3>
<h4 id="代码对象">代码对象</h4>
<p>我们已经看过一个简单的函数的字节码是什么样子的。但是一个典型的 Python 程序要复杂的多。
VM 如何执行一个包含程序定义和函数调用的模块(module)?</p>
<p>考虑如下程序:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nb">print</span><span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
</code></pre></div>
<p>它的字节码什么样子?让我们分析这个程序做了什么来解答这个问题。它定义了一个函数 <code>f</code> ,
通过一个参数 <code>1</code> 调用函数 <code>f</code> 然后打印结果。无论函数 <code>f</code> 做了什么都不会包含在模块字节码中。
我们可以通过运行一个反汇编来证明我们自己:</p>
<div class="highlight"><pre><span></span><code><span class="mf">1</span><span class="w"> </span><span class="mf">0</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_CONST</span><span class="w"> </span><span class="mf">0</span><span class="w"> </span><span class="p">(</span><span class="o"><</span><span class="n">code</span><span class="w"> </span><span class="n">object</span><span class="w"> </span><span class="n">f</span><span class="w"> </span><span class="n">at</span><span class="w"> </span><span class="mf">0</span><span class="n">x10bffd1e0</span><span class="p">,</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="s">"example.py"</span><span class="p">,</span><span class="w"> </span><span class="n">line</span><span class="w"> </span><span class="mf">1</span><span class="o">></span><span class="p">)</span>
<span class="w"> </span><span class="mf">2</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_CONST</span><span class="w"> </span><span class="mf">1</span><span class="w"> </span><span class="p">(</span><span class="err">'</span><span class="n">f</span><span class="err">'</span><span class="p">)</span>
<span class="w"> </span><span class="mf">4</span><span class="w"> </span><span class="n">MAKE_FUNCTION</span><span class="w"> </span><span class="mf">0</span>
<span class="w"> </span><span class="mf">6</span><span class="w"> </span><span class="n">STORE_NAME</span><span class="w"> </span><span class="mf">0</span><span class="w"> </span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="mf">4</span><span class="w"> </span><span class="mf">8</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_NAME</span><span class="w"> </span><span class="mf">1</span><span class="w"> </span><span class="p">(</span><span class="kr">print</span><span class="p">)</span>
<span class="w"> </span><span class="mf">10</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_NAME</span><span class="w"> </span><span class="mf">0</span><span class="w"> </span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="w"> </span><span class="mf">12</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_CONST</span><span class="w"> </span><span class="mf">2</span><span class="w"> </span><span class="p">(</span><span class="mf">1</span><span class="p">)</span>
<span class="w"> </span><span class="mf">14</span><span class="w"> </span><span class="n">CALL_FUNCTION</span><span class="w"> </span><span class="mf">1</span>
<span class="w"> </span><span class="mf">16</span><span class="w"> </span><span class="n">CALL_FUNCTION</span><span class="w"> </span><span class="mf">1</span>
<span class="w"> </span><span class="mf">18</span><span class="w"> </span><span class="n">POP_TOP</span>
<span class="w"> </span><span class="mf">20</span><span class="w"> </span><span class="kr">LOAD</span><span class="n">_CONST</span><span class="w"> </span><span class="mf">3</span><span class="w"> </span><span class="p">(</span><span class="n">None</span><span class="p">)</span>
<span class="w"> </span><span class="mf">22</span><span class="w"> </span><span class="kr">RETURN</span><span class="n">_VALUE</span>
</code></pre></div>
<p>第一行通过从一些叫做代码对象(code object)的东西创建一个函数并绑定名字为 <code>f</code> 来定义函数 <code>f</code> 。
我们没有看到函数 <code>f</code> 用来返回一个自增的参数的字节码。</p>
<p>被作为单一执行单元的代码片段如一个模块或者一个函数体被称为代码块。
CPython 存储关于代码块的信息的结构体就是代码对象(code object)。
它包含字节码和一些其他的比如代码块使用的变量名列表。运行一个模块或者调用一个函数意味着开始
执行相应的代码对象。</p>
<h4 id="函数对象-function-object">函数对象(function object)</h4>
<p>但是,函数不仅仅是代码对象。它必须包含一些额外的信息比如名字、文档字符串(docstring)、
默认参数和定义在闭包作用域中的变量的值。这些信息连同代码对象存储在一个函数对象里。
<code>MAKE_FUNCTION</code> 指令用于创建函数对象。CPython 中定义函数对象的源码前置了如下注释:</p>
<blockquote>
<p>Function objects and code objects should not be confused with each other:</p>
<p>Function objects are created by the execution of the 'def' statement. They reference a code object in their <span class="underline"><span class="underline">code</span></span> attribute, which is a purely syntactic object, i.e. nothing more than a compiled version of some source code lines. There is one code object per source code "fragment", but each code object can be referenced by zero or many function objects depending only on how many times the 'def' statement in the source was executed so far.</p>
</blockquote>
<!--quoteend-->
<blockquote>
<p>函数对象和代码对象不应互相混淆:</p>
<p>函数对象通过执行 'def' 语句创建。它们通过他们的 <span class="underline"><span class="underline">code</span></span> 属性引用一个代码对象,这个代码对象
是一些源代码编译后纯语法对象。每一个代码“片段(fragment)”都对应一个代码对象,
但是每一个代码对象都可以被零个或多个函数对象引用,取决于源码中的 'def' 语句目前为止被执行了多少次。</p>
</blockquote>
<p>多个函数对象如何饮用一个代码对象?这里有个例子:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">make_add_x</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">add_x</span><span class="p">(</span><span class="n">y</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">y</span>
<span class="k">return</span> <span class="n">add_x</span>
<span class="n">add_4</span> <span class="o">=</span> <span class="n">make_add_x</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
<span class="n">add_5</span> <span class="o">=</span> <span class="n">make_add_x</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
</code></pre></div>
<p><code>make_add_x</code> 函数的字节码包含了一个 <code>MAKE_FUNCTION</code> 指令。函数 <code>add_4</code> 和 <code>add_5</code> 是通过同一个
代码对象作为参数调用这个指令产生的结果,但是其参数 <code>x</code> 的值不相同。
每一个函数拥有自己的变量单元的机制允许我们创建如 <code>add_4</code> 和 <code>add_5</code> 的闭包函数。</p>
<p>我们继续下一个主题之前推荐你看一下定义函数对象的代码:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span><span class="w"> </span><span class="nc">PyCodeObject</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">PyObject_HEAD</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_argcount</span><span class="p">;</span><span class="w"> </span><span class="cm">/* #arguments, except *args */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_posonlyargcount</span><span class="p">;</span><span class="w"> </span><span class="cm">/* #positional only arguments */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_kwonlyargcount</span><span class="p">;</span><span class="w"> </span><span class="cm">/* #keyword only arguments */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_nlocals</span><span class="p">;</span><span class="w"> </span><span class="cm">/* #local variables */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_stacksize</span><span class="p">;</span><span class="w"> </span><span class="cm">/* #entries needed for evaluation stack */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_flags</span><span class="p">;</span><span class="w"> </span><span class="cm">/* CO_..., see below */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">co_firstlineno</span><span class="p">;</span><span class="w"> </span><span class="cm">/* first source line number */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_code</span><span class="p">;</span><span class="w"> </span><span class="cm">/* instruction opcodes */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_consts</span><span class="p">;</span><span class="w"> </span><span class="cm">/* list (constants used) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_names</span><span class="p">;</span><span class="w"> </span><span class="cm">/* list of strings (names used) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_varnames</span><span class="p">;</span><span class="w"> </span><span class="cm">/* tuple of strings (local variable names) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_freevars</span><span class="p">;</span><span class="w"> </span><span class="cm">/* tuple of strings (free variable names) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_cellvars</span><span class="p">;</span><span class="w"> </span><span class="cm">/* tuple of strings (cell variable names) */</span>
<span class="w"> </span><span class="n">Py_ssize_t</span><span class="w"> </span><span class="o">*</span><span class="n">co_cell2arg</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Maps cell vars which are arguments. */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_filename</span><span class="p">;</span><span class="w"> </span><span class="cm">/* unicode (where it was loaded from) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">co_name</span><span class="p">;</span><span class="w"> </span><span class="cm">/* unicode (name, for reference) */</span>
<span class="w"> </span><span class="cm">/* ... more members ... */</span>
<span class="p">};</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="k">typedef</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">PyObject_HEAD</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_code</span><span class="p">;</span><span class="w"> </span><span class="cm">/* A code object, the __code__ attribute */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_globals</span><span class="p">;</span><span class="w"> </span><span class="cm">/* A dictionary (other mappings won't do) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_defaults</span><span class="p">;</span><span class="w"> </span><span class="cm">/* NULL or a tuple */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_kwdefaults</span><span class="p">;</span><span class="w"> </span><span class="cm">/* NULL or a dict */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_closure</span><span class="p">;</span><span class="w"> </span><span class="cm">/* NULL or a tuple of cell objects */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_doc</span><span class="p">;</span><span class="w"> </span><span class="cm">/* The __doc__ attribute, can be anything */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_name</span><span class="p">;</span><span class="w"> </span><span class="cm">/* The __name__ attribute, a string object */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_dict</span><span class="p">;</span><span class="w"> </span><span class="cm">/* The __dict__ attribute, a dict or NULL */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_weakreflist</span><span class="p">;</span><span class="w"> </span><span class="cm">/* List of weak references */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_module</span><span class="p">;</span><span class="w"> </span><span class="cm">/* The __module__ attribute, can be anything */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_annotations</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Annotations, a dict or NULL */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">func_qualname</span><span class="p">;</span><span class="w"> </span><span class="cm">/* The qualified name */</span>
<span class="w"> </span><span class="n">vectorcallfunc</span><span class="w"> </span><span class="n">vectorcall</span><span class="p">;</span>
<span class="p">}</span><span class="w"> </span><span class="n">PyFunctionObject</span><span class="p">;</span>
</code></pre></div>
<h4 id="帧对象-frame-object">帧对象(frame object)</h4>
<p>当执行一个代码对象时,VM 需要一直跟踪变量的值并不断的更新值栈(value stack)。
同时还需要记住在什么地方停止运行当前代码对象然后去运行其他的代码对象,并且在哪里返回。
CPython 在一个帧对象里存储这些信息,或者简单的说成帧。一个帧提供了一个哪个代码对象可以被执行的状态。
由于我们已经开始习惯源代码,所以这里我贴出帧对象的定义:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span><span class="w"> </span><span class="nc">_frame</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">PyObject_VAR_HEAD</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">_frame</span><span class="w"> </span><span class="o">*</span><span class="n">f_back</span><span class="p">;</span><span class="w"> </span><span class="cm">/* previous frame, or NULL */</span>
<span class="w"> </span><span class="n">PyCodeObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_code</span><span class="p">;</span><span class="w"> </span><span class="cm">/* code segment */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_builtins</span><span class="p">;</span><span class="w"> </span><span class="cm">/* builtin symbol table (PyDictObject) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_globals</span><span class="p">;</span><span class="w"> </span><span class="cm">/* global symbol table (PyDictObject) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_locals</span><span class="p">;</span><span class="w"> </span><span class="cm">/* local symbol table (any mapping) */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">**</span><span class="n">f_valuestack</span><span class="p">;</span><span class="w"> </span><span class="cm">/* points after the last local */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">**</span><span class="n">f_stacktop</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Next free slot in f_valuestack. ... */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_trace</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Trace function */</span>
<span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="n">f_trace_lines</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Emit per-line trace events? */</span>
<span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="n">f_trace_opcodes</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Emit per-opcode trace events? */</span>
<span class="w"> </span><span class="cm">/* Borrowed reference to a generator, or NULL */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_gen</span><span class="p">;</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">f_lasti</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Last instruction if called */</span>
<span class="w"> </span><span class="cm">/* ... */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">f_lineno</span><span class="p">;</span><span class="w"> </span><span class="cm">/* Current line number */</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">f_iblock</span><span class="p">;</span><span class="w"> </span><span class="cm">/* index in f_blockstack */</span>
<span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="n">f_executing</span><span class="p">;</span><span class="w"> </span><span class="cm">/* whether the frame is still executing */</span>
<span class="w"> </span><span class="n">PyTryBlock</span><span class="w"> </span><span class="n">f_blockstack</span><span class="p">[</span><span class="n">CO_MAXBLOCKS</span><span class="p">];</span><span class="w"> </span><span class="cm">/* for try and loop blocks */</span>
<span class="w"> </span><span class="n">PyObject</span><span class="w"> </span><span class="o">*</span><span class="n">f_localsplus</span><span class="p">[</span><span class="mi">1</span><span class="p">];</span><span class="w"> </span><span class="cm">/* locals+stack, dynamically sized */</span>
<span class="p">};</span>
</code></pre></div>
<p>第一个帧被创建用来执行模块的代码对象。任何时候需要执行另外一个代码对象 CPython 都会创建创建新的
帧去执行该代码对象。每一个帧都有一个引用指向前一个帧。从而,帧形成了一个栈被称为调用栈,当前帧位于
顶部。当一个函数被调用,一个新的帧被压到栈上。当从当前执行帧返回时,CPython 通过记录的最后处理的指令
来继续执行前一个帧。某种意义上 CPython 除了执行帧其他什么也没做。但是接下来我们马上看到这个总结
善意的隐藏了某些细节。</p>
<h3 id="线程-解释器-运行时-threads-interpreters-runtime">线程、解释器、运行时(Threads, interpreters, runtime)</h3>
<p>我们已经讨论过三个重要的主题:</p>
<ul>
<li>代码对象</li>
<li>函数对象,和</li>
<li>帧对象</li>
</ul>
<p>CPython 还有三个重要的主题:</p>
<ul>
<li>线程状态(thread stage)</li>
<li>解释器状态(interpreter state),和</li>
<li>运行时状态(runtime state)</li>
</ul>
<h4 id="线程状态">线程状态</h4>
<p>线程状态是一个包含线程特定数据栈数据结构,其中包含调用栈、异常状态和调试设置。
不应将其和系统线程(OS thread)混淆,尽管它们联系紧密。考虑当时使用标准库 <code>threading</code>
在一个单独的线程运行一个函数发生了什么:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">threading</span> <span class="kn">import</span> <span class="n">Thread</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">():</span>
<span class="w"> </span><span class="sd">"""Perform an I/O-bound task"""</span>
<span class="k">pass</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">)</span>
<span class="n">t</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">t</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre></div>
<p><code>t.start</code> 实际上创建通过系统调用(类 Unix 系统中通过 <code>pthread_create</code> ,Windows 通过
<code>_beginthreadex</code> )了一个新的系统线程。新建的线程调用在 <code>_thread</code> 模块中的函数负责调
用相应的目标函数。这个函数不仅仅接收目标函数和目标函数的参数,同时一个新的线程状态
被用在了新建的系统线程上。系统线程通过它自己的线程状态进入执行循环,并一直持有。</p>
<p>这里我们可能记得阻止多线程同时陷入执行循环的著名的 GIL(Global Interpreter Lock)。
主要原因是为了在不引入更多细粒度的锁的情况下保护 CPython 状态免受损坏。
<a href="https://docs.python.org/zh-cn/3.9/c-api/init.html#thread-state-and-the-global-interpreter-lock">Python C/API 参考</a>清晰的解释了 GIL:</p>
<blockquote>
<p>The Python interpreter is not fully thread-safe. In order to support multi-threaded Python programs, there’s a global lock, called the global interpreter lock or GIL, that must be held by the current thread before it can safely access Python objects. Without the lock, even the simplest operations could cause problems in a multi-threaded program: for example, when two threads simultaneously increment the reference count of the same object, the reference count could end up being incremented only once instead of twice.</p>
</blockquote>
<!--quoteend-->
<blockquote>
<p>Python 解释器不是完全的线程安全。为了支持 Python 多线程程序,引入一个称为全局解释器锁或者 GIL 的全局锁,
当前线程必须持有该锁才能安全的访问 Python 对象。如果没有持有该锁,就连最简单的操作都会在多线程程序中引发
问题:比如,当两个线程同时增加同一个对象的引用计数,引用计数最终可能只被增加了一次。</p>
</blockquote>
<p>要管理多线程,就需要一个比线程状态更高层级的数据结构。</p>
<h4 id="解释器状态和运行时状态-interpreter-and-runtime-states">解释器状态和运行时状态(interpreter and runtime states)</h4>
<p>实际上,这是两个状态:解释器状态和运行时状态。两者的需求区分看起来似乎不明显。但是,任何程序
的执行都需要各个状态的最少一个实例,并且有合理的原因。</p>
<p>解释器状态是一组线程以及该组相关的数据。线程共享诸如加载的模块(sys.modules)、内建对象(builtins.__dict__)
和导入系统(importlib)。</p>
<p>运行时状态是一个全局变量。保存着进程相关的数据。包含 CPython 状态(是否初始化)和 GIL。</p>
<p>通常情况下,一个进程的所有线程都属于同一个解释器。但是,有一些罕见的情况比如有人想创建一个子解释器来隔离一组线程。
比如 <a href="https://modwsgi.readthedocs.io/en/develop/user-guides/processes-and-threading.html#python-sub-interpreters">mod_wsgi</a> 使用不同的解释器来运行 WSGI 程序。最明显的隔离效果是各组线程拥有它们自己版本的模块,包括 <code>__main__</code> ,
也就是隔离全局命名空间(global namespace)。</p>
<p>CPython 没有提供像 <code>threading</code> 模块那样简单的方式创建新的解释器。这个特性仅仅通过 Python/C API 提供支持,
但是<a href="https://www.python.org/dev/peps/pep-0554/">未来有可能改善</a>。</p>
<h3 id="架构摘要-architecture-summary">架构摘要(Architecture summary)</h3>
<p>让我们来快速总结 CPython 的架构来看看这一切是如何组织在一块的。解释器可以被看作分层结构(layer structure)。
这些层级包括:</p>
<ol>
<li>运行时(Runtime):进程的全局 CPython 状态;包含 GIL 和内存分配机制。</li>
<li>解释器(Interpreter):一组线程和它们共享的数据,如导入的模块。</li>
<li>线程(Thread):特定于单个系统线程的数据;包含调用栈。</li>
<li>帧(Frame):调用栈的元素;提供执行一个代码对象的状态。</li>
<li>执行循环(Evalution loop):执行一个代码对象(描述代码块做了什么,包含字节码、变量名字)</li>
</ol>
<p>我们已经看到不同的层级通过相应的数据结构来表示。在某些情况下它们很难等效。比如,内存分配机制
通过使用全局变量来实现。这不是运行时状态的一部分,但是绝对是 CPython 运行时层级的一部分。</p>
<h3 id="总结-conclusion">总结(Conclusion)</h3>
<p>在这一部分我们已经大体描述了 <code>python</code> 在执行一个 Python 程序时做了什么。我们已经看到它工作在三个状态:</p>
<ol>
<li>初始化 Python 运行时</li>
<li>编译源代码到一个模块代码对象;然后</li>
<li>执行代码对象的字节码。</li>
</ol>
<p>解释器中负责执行字节码的部分称为虚拟机(VM,virtual machine)。CPython VM 包含一些特别重要的概念:
代码对象(code object)、帧对象(frame object)、线程状态(thread state)、解释器状态(interpreter state)和
运行时(runtime)。这些数据结构构成了 CPython 架构的核心。</p>
<p>我们还有很多内容没有涉及到。我们避免陷入到源代码中。初始化和编译阶段完全超出了我们的范围。
相反,我们从虚拟机的概述开始。通过这种方式,我认为,我们可以更好的看到每个阶段所负责的内容。
现在我们知道 CPython 将源代码编译成了什么 -- 代码对象(code object)。
接下来我们将看到它是如何做到的。</p>
<p>如果你有任何问题、评论或者建议,随时通过 victor@tenthousandmeters.com 联系原作者。</p>
<p><strong><em>*Update 4 September 2020</em></strong>*: I've made <a href="https://tenthousandmeters.com/materials/python-behind-the-scenes-a-list-of-resources/">a list of resources</a> that I've used to learn about CPython internals</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:fn:1">
<p>译注: <code>python -c "import dis;dis.dis('''def g(x): return x + 3''')"</code> <a class="footnote-backref" href="#fnref:fn:1" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
</ol>
</div>【译】Rust 意味着无需手动关闭 Socket 连接2020-08-27T00:00:00+08:002020-08-27T00:00:00+08:00coldtag:www.linuxzen.com,2020-08-27:/rust-means-never-having-to-close-a-socket.html<blockquote>
<p>译者注:这是我学习 Rust 生命周期对我最有帮助的文章之一,故翻译了一下。</p>
</blockquote>
<p>原文链接:<a href="https://blog.skylight.io/rust-means-never-having-to-close-a-socket/">Rust Means Never Having to Close a Socket</a></p>
<p>Rust 最酷的特性之一 …</p><blockquote>
<p>译者注:这是我学习 Rust 生命周期对我最有帮助的文章之一,故翻译了一下。</p>
</blockquote>
<p>原文链接:<a href="https://blog.skylight.io/rust-means-never-having-to-close-a-socket/">Rust Means Never Having to Close a Socket</a></p>
<p>Rust 最酷的特性之一就是它可以自动地帮助你管理资源,同时在仍能保持安全(没有段错误)和高性能。</p>
<p>这是因为 Rust 是一门与众不同地编程语言,要理解我说的可能有点困难,让我来更近一步说明:</p>
<ul>
<li>Rust 就像带垃圾回收的编程语言,你无需手动释放内存</li>
<li>Rust 不同于其他带垃圾回收的编程语言,你无需<sup id="fnref:fn:1"><a class="footnote-ref" href="#fn:fn:1">1</a></sup>手动关闭或者释放像文件、套接字和锁这样的资源</li>
<li>Rust 达到以上这些特性不附带任何运行时开销(垃圾回收或者引用计数),并且不牺牲安全性。</li>
</ul>
<p>如果你曾经造成过一个套接字或者文件泄漏,或者使用过一些抽象方法造成了这些资源的泄漏,那么你就会知道这有多重要。</p>
<p>你可能已经期望通过“使用后释放”来避免内存问题,而与此同时你并没有考虑到没有明确地关闭套接字可能出现类似的错误。我在这里告诉你,还有更好地办法。</p>
<p>如果你使用的是带垃圾回收的编程语言,则应密切关注本文提到的资源管理方面的内容。如果你使用的是像 C/C++ 这样底层编程语言,你可能会对安全方面更加感兴趣。</p>
<blockquote>
<p>Rust 的许多特性都是从其他语言借鉴而来。Rust 之所以变得有趣是因为它把所有的这些特性放在了一起,并且在编程语言层面实现了更严格地保证。
实际上,这种编程语言层面的保证让这些特性更加实用。</p>
</blockquote>
<h2 id="所有权系统-the-ownership-system">所有权系统(The Ownership System)</h2>
<p>让这种保证工作的方式是通过 Rust 的「所有权(ownership)」系统。不管任何时候你创建一个新的对象,都被创建它的「作用域(scope)」所拥有。</p>
<p>让我们通过一个例子来进一步说明:我们定义一个函数,函数拷贝输入文件到临时文件去处理它,然后拷贝输入文件到输出文件。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">process</span><span class="p">(</span><span class="n">from</span>: <span class="kp">&</span><span class="nc">Path</span><span class="p">,</span><span class="w"> </span><span class="n">to</span>: <span class="kp">&</span><span class="nc">Path</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">IoResult</span><span class="o"><</span><span class="p">()</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// creates a new tempdir with the specified suffix</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">tempdir</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kr">try</span><span class="o">!</span><span class="p">(</span><span class="n">TempDir</span>::<span class="n">new</span><span class="p">(</span><span class="s">"skylight"</span><span class="p">));</span>
<span class="w"> </span><span class="c1">// open the input file</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">from_file</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kr">try</span><span class="o">!</span><span class="p">(</span><span class="n">File</span>::<span class="n">open</span><span class="p">(</span><span class="n">from</span><span class="p">));</span>
<span class="w"> </span><span class="c1">// create a temporary file inside the tempdir</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">tempfile</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="kr">try</span><span class="o">!</span><span class="p">(</span><span class="n">File</span>::<span class="n">create</span><span class="p">(</span><span class="o">&</span><span class="n">tempdir</span><span class="p">.</span><span class="n">path</span><span class="p">().</span><span class="n">join</span><span class="p">(</span><span class="s">"tmp1"</span><span class="p">)));</span>
<span class="w"> </span><span class="c1">// copy the input file into the tempfile</span>
<span class="w"> </span><span class="kr">try</span><span class="o">!</span><span class="p">(</span><span class="n">io</span>::<span class="n">util</span>::<span class="n">copy</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">from_file</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">tempfile</span><span class="p">));</span>
<span class="w"> </span><span class="c1">// use an external program to process the tmpfile in place</span>
<span class="w"> </span><span class="c1">// after processing, copy the tempfile into the output file</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kr">try</span><span class="o">!</span><span class="p">(</span><span class="n">File</span>::<span class="n">create</span><span class="p">(</span><span class="n">to</span><span class="p">));</span>
<span class="w"> </span><span class="n">io</span>::<span class="n">util</span>::<span class="n">copy</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">tempfile</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">out</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div>
<p>在这个例子中,函数 <code>process</code> 的作用域再第一行创建了 <code>TempDir</code> 是其初始拥有者。在这个例子中, <code>process</code> 函数从未放弃所有权,所以当函数完成调用,
它就会自动被丢弃(dropped),也就是会删除 <code>Tempfile</code> 。</p>
<p>这就是一个关于自动资源管理的例子。 <code>TempDir</code> 对象不仅仅是一片内存,它还代表被管理的资源。一旦程序不在使用该资源,那么它的清理逻辑将会被调用。</p>
<blockquote>
<p>另外:在这 C++ 中被称为 「RAII」(Resource Acquistion Is Initialization):资源获取即初始化,它是编程中最容易混淆但是有用的命名。</p>
</blockquote>
<p>对我来说很有趣地是,能最大效率地减轻程序员手动管理内存的技术往往也最难成功和有效地减轻程序员手动地管理资源。
在高级语言中,我们从不需要释放内存,但是我们通常需要关闭套接字、文件和释放锁。</p>
<p>在实际中,在带有垃圾回收机制的编程语言中泄漏这些资源的情况令人震惊,所以我真的很享受这样一个事实,
在 Rust 中忘记关闭套接字不是一个大问题,就像在 Rust 忘记释放内存一样。并且在 Rust 中,
你可以免受防御涉及资源的“释放后使用”错误,就像你免受防御涉及内存的“释放后使用”错误一样。</p>
<p>这听起来很神奇,所以你可能会有一些问题关于它实际上是如何工作的。</p>
<p>首先,这个系统基于事实上一个对象在同一时间只能有一个所有者。我该如何确保我没有错误地在多个地方引用 <code>TempDir</code> ?
答案是所有权系统不是建议性的。在 Rust 中,对象被创建其的作用域所拥有。它可以将所有权转移到其他作用域,或者在完成执行后保留所有权。
当一个作用域完成时,Rust 将销毁它所拥有地所有对象。</p>
<p>因为一个对象同时只能归一个作用域所有,你可以通过查看就知道执行结束时有哪些对象将被销毁。</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="nb">String</span><span class="p">,</span>
<span class="w"> </span><span class="n">last</span>: <span class="nb">String</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">hello</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">yehuda</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="s">"Yehuda"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">last</span>: <span class="s">"Katz"</span><span class="p">.</span><span class="n">to_string</span><span class="p">()</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// `yehuda` is transferred to `name_size`, so it cannot be</span>
<span class="w"> </span><span class="c1">// used anymore in this function, and it will not be destroyed</span>
<span class="w"> </span><span class="c1">// when this function returns. It is up to `name_size`,</span>
<span class="w"> </span><span class="c1">// or possibly a future owner, to destroy it.</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">name_size</span><span class="p">(</span><span class="n">yehuda</span><span class="p">);</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">tom</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="s">"Tom"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">last</span>: <span class="s">"Dale"</span><span class="p">.</span><span class="n">to_string</span><span class="p">()</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// `tom` wasn't transferred, so it will be</span>
<span class="w"> </span><span class="c1">// destroyed when this function returns.</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">name_size</span><span class="p">(</span><span class="n">person</span>: <span class="nc">Person</span><span class="p">)</span><span class="w"> </span>-> <span class="nc">uint</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">first</span><span class="p">,</span><span class="w"> </span><span class="n">last</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">person</span><span class="p">;</span>
<span class="w"> </span><span class="n">first</span><span class="p">.</span><span class="n">len</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">last</span><span class="p">.</span><span class="n">len</span><span class="p">()</span>
<span class="w"> </span><span class="c1">// this function owns Person, so the Person is destroyed when `name_size` returns</span>
<span class="p">}</span>
</code></pre></div>
<p>仅仅通过逐一查看这两个函数,你可以看到 <code>yehuda</code> 被转移到了 <code>name_size</code> 函数,但是 <code>tom</code> 则没有。
通过查看 <code>name_size</code> 函数,你可以看到它一直拥有它的 <code>person</code> 参数直到它返回。仅仅通过查看这两个函数,
你就可以直接确定哪个对象(如果有)将会在它们执行完毕被销毁。</p>
<p>但是如何解释临时文件的例子?如果你查看 <code>process</code> 函数的第三行代码,你可以看到 <code>TempDir</code> 上的方法 <code>tempdir.path()</code> 被调用。
难道这不是意味着我创建了第二个引用,并且理论上有两个所有者?或者意味着我们将所有权转移到了 <code>path</code> 方法,也就是当该方法返回时会立即销毁这个目录?
显然这两个答案都行不通。</p>
<h2 id="借用和借出-borrowing-and-lending">借用和借出(Borrowing and Lending)</h2>
<p>要理解这里发生了什么,我们需要看一下 <code>path</code> 方法的方法签名。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">path</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="nc">Path</span>
</code></pre></div>
<p>可以通过如下方式念出这个方法签名:</p>
<blockquote>
<p>path 方法「借用(borrows)」self 并返回「借用的(borrowed)」Path。</p>
</blockquote>
<p>一个函数借用一个对象并不会取的对象的所有权,并且在返回时不会销毁该对象。它只能在函数调用期间使用借用的对象,它不能,比如,创建线程并在线程中使用借用的对象。
换句话说,借用的对象必须不能在超出借出它的函数的作用域外存活。</p>
<p>这意味着 Rust 编译器会检查所有函数调用并且在编译期得知代码是否尝试获取所有权。一旦一个对象的所有权被转移,那么原所有者会被拒绝访问该对象。</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="nb">String</span><span class="p">,</span>
<span class="w"> </span><span class="n">last</span>: <span class="nb">String</span><span class="p">,</span>
<span class="w"> </span><span class="n">age</span>: <span class="nc">uint</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">hello</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">person</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="s">"Yehuda"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">last</span>: <span class="s">"Katz"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">age</span>: <span class="mi">32</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">thirties</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">is_thirties</span><span class="p">(</span><span class="n">person</span><span class="p">);</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"{}, thirties: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">person</span><span class="p">,</span><span class="w"> </span><span class="n">thirties</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// This function tries to take ownership of `Person`; it does not</span>
<span class="c1">// ask to borrow it by taking &Person</span>
<span class="k">fn</span> <span class="nf">is_thirties</span><span class="p">(</span><span class="n">person</span>: <span class="nc">Person</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">person</span><span class="p">.</span><span class="n">age</span><span class="w"> </span><span class="o">>=</span><span class="w"> </span><span class="mi">30</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">person</span><span class="p">.</span><span class="n">age</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">40</span>
<span class="p">}</span>
</code></pre></div>
<p>如果我尝试编译这段代码,我会得到下面的编译错误(略有删节):</p>
<div class="highlight"><pre><span></span><code><span class="n">move</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">16</span><span class="o">:</span><span class="mi">34</span><span class="o">:</span><span class="w"> </span><span class="mi">16</span><span class="o">:</span><span class="mi">40</span><span class="w"> </span><span class="k">error</span><span class="o">:</span><span class="w"> </span><span class="k">use</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">moved</span><span class="w"> </span><span class="k">value</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`person`</span>
<span class="n">move</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">16</span><span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s2">"{}, thirties: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">person</span><span class="p">,</span><span class="w"> </span><span class="n">thirties</span><span class="p">);</span>
<span class="w"> </span><span class="o">^~</span><span class="err">\</span><span class="o">~~~~</span>
<span class="n">move</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">15</span><span class="o">:</span><span class="mi">32</span><span class="o">:</span><span class="w"> </span><span class="mi">15</span><span class="o">:</span><span class="mi">38</span><span class="w"> </span><span class="n">note</span><span class="o">:</span><span class="w"> </span><span class="n n-Quoted">`person`</span><span class="w"> </span><span class="n">moved</span><span class="w"> </span><span class="n">here</span>
<span class="n">move</span><span class="p">.</span><span class="n">rs</span><span class="o">:</span><span class="mi">15</span><span class="w"> </span><span class="n">let</span><span class="w"> </span><span class="n">thirties</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">is_thirties</span><span class="p">(</span><span class="n">person</span><span class="p">);</span>
<span class="w"> </span><span class="o">^~</span><span class="err">\</span><span class="o">~~~~</span>
</code></pre></div>
<p>错误的意思是 <code>hello</code> 函数的作用域是 <code>Person</code> 的初始所有者,但是当调用 <code>is_thirties</code> 时,它把所有权转移到了 <code>is_thirties</code> 函数的作用域。
作为新的所有者,当 <code>is_thirties</code> 返回,它就会释放 <code>Person</code> 占据的内存。</p>
<p>作为替代你会想使用「借用和借出」写这个程序:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">hello</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">person</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="s">"Yehuda"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">last</span>: <span class="s">"Katz"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">age</span>: <span class="mi">32</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// lend the person -- don't transfer ownership</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">thirties</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">is_thirties</span><span class="p">(</span><span class="o">&</span><span class="n">person</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// now this scope still owns the person</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"{}, thirties: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">person</span><span class="p">,</span><span class="w"> </span><span class="n">thirties</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">is_thirties</span><span class="p">(</span><span class="n">person</span>: <span class="kp">&</span><span class="nc">Person</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">person</span><span class="p">.</span><span class="n">age</span><span class="w"> </span><span class="o">>=</span><span class="w"> </span><span class="mi">30</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">person</span><span class="p">.</span><span class="n">age</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">40</span>
<span class="p">}</span>
</code></pre></div>
<p><strong><em>*从根本上讲,这意味着经过验证地所有权是你函数接口的一部分。</em></strong>* Rust 开发者有时将其称为“借用检查器(borrow checker)”,但是却恰当好处。</p>
<p>实际上,这些大部分时间可以正常工作的原因是,函数获得它们值的方式是通过“借用(borrowing)”。它们获得值、通过这些值处理一些逻辑然后返回。
长时间保持该值(比如通过使用线程)既不常见,又是时候该考虑一下正在发生的什么。</p>
<p>当我们开始编写一个新的函数时应该借用所需参数,而不是尝试获取其所有权。经过一段时间的 Rust 编程之后这将不会增加认知成本,只是默认这样做。
如果编译器抱怨(随着你掌握这些规则将它们变成习惯(second nature)这样的情况将越来越少),这意味着你正在做含有潜在危险地事情,那就需要你思考一下了。</p>
<h2 id="从一个借用对象中返回一个借用字段-returning-a-borrowed-field-from-a-borrowed-object">从一个借用对象中返回一个借用字段(Returning a Borrowed Field from a Borrowed Object)</h2>
<p>在前面我们检查了如下方法签名:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">path</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="nc">Path</span>
</code></pre></div>
<p>这个签名可能让你困惑。我之前说过当一个函数借用了一个对象,它必须只能在函数调用期间使用这个值,并且在此之后就不能使用。
难道返回对象的其中一部分没有违反这个规则?</p>
<p>这个之所以没问题是因为 <code>path</code> 的调用者明显有权使用 <code>Tempfile</code> 并通过参数将之借给 <code>path</code> 。
在这个案例中,Rust 编译器将会保证返回的 <code>Path</code> 没有在超出拥有 <code>Tempfile</code> 的作用域之外存活。</p>
<p>实际上,这意味着你可以返回从上游借来的内容,然后 Rust 将处理好跟踪该内容的原容器。</p>
<p>让我们通过一个例子来举例说明:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">hello</span><span class="p">()</span><span class="w"> </span>-> <span class="kp">&</span><span class="kt">str</span> <span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">person</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Person</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">first</span>: <span class="s">"Yehuda"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">last</span>: <span class="s">"Katz"</span><span class="p">.</span><span class="n">to_string</span><span class="p">(),</span>
<span class="w"> </span><span class="n">age</span>: <span class="mi">32</span>
<span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="n">first_name</span><span class="p">(</span><span class="o">&</span><span class="n">person</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">first_name</span><span class="p">(</span><span class="n">person</span>: <span class="kp">&</span><span class="nc">Person</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="kt">str</span> <span class="p">{</span>
<span class="w"> </span><span class="c1">// as_slice borrows a slice "view" out of a string</span>
<span class="w"> </span><span class="n">person</span><span class="p">.</span><span class="n">first</span><span class="p">.</span><span class="n">as_slice</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div>
<p>如果你仔细观察,你可以立即看到问题所在。函数 <code>hello</code> 试图返回一个借用的 <code>&str</code> ,但是拥有包含被返回的字节的原 <code>Person</code> 的所有权在 <code>hello</code> 中。
一旦 <code>hello</code> 返回,那么 <code>Person</code> 将不复存在,导致借用的内容(字符串切片)指向了无效地地址。</p>
<p>如果试图编译这段代码,你将得到如下报错:</p>
<div class="highlight"><pre><span></span><code><span class="n">move</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">8</span><span class="err">:</span><span class="mi">15</span><span class="err">:</span><span class="w"> </span><span class="mi">8</span><span class="err">:</span><span class="mi">19</span><span class="w"> </span><span class="nl">error</span><span class="p">:</span><span class="w"> </span><span class="n">missing</span><span class="w"> </span><span class="n">lifetime</span><span class="w"> </span><span class="n">specifier</span><span class="w"> </span><span class="o">[</span><span class="n">E0106</span><span class="o">]</span>
<span class="n">move</span><span class="p">.</span><span class="nl">rs</span><span class="p">:</span><span class="mi">8</span><span class="w"> </span><span class="n">fn</span><span class="w"> </span><span class="n">hello</span><span class="p">()</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="o">&</span><span class="nf">str</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="o">^~</span><span class="err">\</span><span class="o">~~</span>
</code></pre></div>
<p>这个有点混乱地错误信息表示我们正尝试返回借用的字节,但是函数的调用者没有借给我们借用字节的来源 <code>Person</code> 。
Rust 正在向我们征询如果返回值不属于调用者作用域那么应该归属于那个「生命周期(lifetime)」。</p>
<blockquote>
<p>通常情况下,Rust 会将返回值的作用域绑定到借用参数的作用域。这在里,我们没有借用的参数,所以 Rust 要求我们进行显式地定义。</p>
</blockquote>
<p>实际情况下,这表示一个函数可以轻松地通过借用的方式返回借用参数中的内容。否则,你需要给该返回值找到一个调用者可以访问的存储位置,
或者克隆(clone)该值让调用者拥有一份自己的拷贝。</p>
<h2 id="易用性-ergonomics">易用性(Ergonomics)</h2>
<p>咋一看,所有权这些机制让人感觉很复杂,并且看起来可能会对使用 Rust 的易用性产生重大影响。可以肯定的是,一开始确实会有这种感觉。</p>
<p>但是有几个因素会让 Rust 的所有权变得比远看上去更加易用。</p>
<p>首先,大量的实际代码适用于借用/借出模式。随着我写了越来越多的 Rust 代码,我逐渐意识到用 Ruby 编写的程序遵循类似的模式:
函数创建一些对象并将它们传递给子函数执行某些任务,然后子函数返回新值。</p>
<p>当然,这是递归的,因此仅当差异(在于函数调用期间使用参数,以及延长参数的使用周期)在 Rust 中被显式区分时,它才能变得显而易见。
只有通过函数签名进行区分和全面覆盖,并检查错误,我们才能获得 Rust 提供的保证。</p>
<blockquote>
<p>相反,C++ 只对部分情况进行了明显地区分,并且没有错误检查。带有垃圾回收机制的语言通常会隐藏“转移(transferred)”和“借出(lent)”参数。</p>
</blockquote>
<p>如我上面所说,这意味着 Rust 程序员快速学习将借用作为默认行为去编写函数来减轻许多系统认知负担。</p>
<p>其次,随着使用一段时间 Rust 之后,大部分人意识到借用检查器错误和警告他们的都是真实地、严重地和比较难以发觉的错误。
一段时间以后,借用检查器自然地将您推入编程模式并且减少出现此类难以发觉地错误的影响。</p>
<p>再次,我个人发现,对所有权的清晰了解可以大大提高我对程序进行推理地能力,同时避免意外引入后面会花费大量地时间跟踪排查的内存泄漏问题。</p>
<p>最后,自动资源管理具有真正地易用性优势,既可以防止资源泄漏(当我懒惰时),又可以防止额外地样板代码和缩进(当我谨慎时)。</p>
<p>除 C++ 之外,很少程序员经历过自动资源管理为标准的编程环境,打开大脑的“blub”部分并认为它没有那么大的作用是非常非常容易的。
Rust 改变了这个领域中的传统权衡,我建议您将脑中告诉自己「我不需要它在《我选择的编程语言》,它有多重要? ?」的小声音暂时先放回去。</p>
<h2 id="引用计数-和垃圾回收">引用计数(和垃圾回收)</h2>
<p>你可能已经注意到 Rust 已经有引用计数指针(并且计划未来实现 GC)。</p>
<p>它是如何在所有权系统中工作的呢?</p>
<p>以我的经验,一旦习惯了所有权范式,你会很少想要使用 <code>Rc</code> 指针。例如,整个 Cargo 代码库中没有使用引用计数指针的实例,
只使用了一次原子计数指针(用于在并行构建的代码的线程之间共享锁)。</p>
<p>我认为这是由于所有权非常明确,并且切实地改善了本地推理。如果你检查任意使用正常 Rust 引用的函数,
则可以轻易地知道一旦函数返回哪些内存(和资源)仍将存活,而哪些不会。例如,如果你使用闭包,
则可以立即判断它是否存活于当前函数作用域外,如果它确实存活于当前函数作用域外,你还可以知道闭包拥有哪些对象。</p>
<p>我也认为所有权和借出概念可以很好地映射到实际地编程模式。有一些事情你不可以做,但是大部分情况下,略微调整代码结构就可以通过编译。
作为交换,内存和资源泄漏都很少发生,并且代码清晰度得到了提高。</p>
<p>如果不是这种情况,我怀疑即使是经验丰富地 Rust 开发人员也会更频繁地使用 <code>Rc</code> 。</p>
<p>综上所述,在某些情况下,引用计数甚至垃圾回收也可以正常的在所有权系统下工作。 Rust 的 “智能指针” 系统允许 <code>Rc</code> 指针在相同地所有权和借用系统内透明地运行,
并且当引用计数减小到 0 时运行析构函数(伴随明显地本地推理和运行时性能上的成本)。</p>
<h2 id="其他语言的机制-facilities-in-other-languages">其他语言的机制(Facilities in Other Languages)</h2>
<p>带有垃圾回收机制的语言通常会提供一些机器辅助程序员手动管理资源。在大部分现代编程语言中,你不用显式的调用 <code>close</code> ,
但是你需要调整语言结构将资源与词法作用域联系在一起,然后在完成后进行释放。</p>
<p>让我们观察一些例子,然后我会讨论这些方法的缺点。</p>
<p>在 Ruby 中,你可以使用一个块标识你将在指定的作用域里使用资源。一旦块返回,资源将会被清理。</p>
<div class="highlight"><pre><span></span><code><span class="no">File</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"/etc/passwd"</span><span class="p">)</span><span class="w"> </span><span class="k">do</span><span class="w"> </span><span class="o">|</span><span class="n">file</span><span class="o">|</span>
<span class="w"> </span><span class="c1"># use the file</span>
<span class="k">end</span>
</code></pre></div>
<p>在 Python 中,一个特殊的语言关键字 <code>with</code> 用来创建一个协议进行资源获取,然后在代码块结束后释放资源:</p>
<div class="highlight"><pre><span></span><code><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"/etc/passwd"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
<span class="c1"># use the file</span>
</code></pre></div>
<p>Ruby 和 Python 都使用了通过调整语言结构和创建新协议的方法抽象了特定于资源关闭的机制。用户永远不知道关闭什么样子,
但是他们必须使用特殊的抽象来确保资源关闭被调用。</p>
<p>在 Go 中, <code>defer</code> 关键字允许程序员在原始创建逻辑之后提供清理逻辑来管理资源:</p>
<div class="highlight"><pre><span></span><code><span class="nx">file</span><span class="p">,</span><span class="w"> </span><span class="kt">error</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">os</span><span class="p">.</span><span class="nx">Open</span><span class="p">(</span><span class="s">"/etc/passwd"</span><span class="p">)</span>
<span class="k">if</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">nil</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">defer</span><span class="w"> </span><span class="nx">file</span><span class="p">.</span><span class="nx">Close</span><span class="p">()</span>
<span class="c1">// use the file</span>
</code></pre></div>
<p>这种方式比 <code>try/catch/finally</code> 有好一些,因为它保持清理逻辑紧跟资源获取逻辑,但是没有抽象关闭逻辑。</p>
<p>所有这些方法都有一系列的问题。再次,我建议你远离大脑中的很可能会告诉您这些问题“在实践中不会变得很重要”的“blub”中心。</p>
<ul>
<li>向已经使用中的结构后面添加资源释放逻辑是不可能的,因为他们的使用者(clients)将会使用正常的对象创建 API。
这会导致在更高层对象中抽象资源更加困难,因为资源管理需要暴漏到公共 API。</li>
<li>基于块的方法(Ruby 和 Python,不包含 Go)引入向右漂移。每次你想要使用一个资源,你都被强制创建一个新的作用域范围。
这在 Ruby(有很好的块)和 Python(使用语言层面的结构)中相当烦人,在 JavaScript 中还有一个严重地问题,即引入新的作用域会阻止你返回或者摆脱当前循环。</li>
<li>这些方法(包含 Go 的 <code>defer</code> )需要你在给定的词法作用域内使用资源。当你想要将资源传递给多个函数时将会引发尴尬(或不可能的)编程风格。
实际上,它迫使语言使用一个不地道的基于作用域的所有权系统模型进行对象管理。<ul>
<li>一旦你开始使用资源调用其他函数,则可能意外地创建“释放后使用” 的 bug,如函数绑定在资源上(如闭包中),并在调用者关闭资源后尝试使用它。</li>
</ul>
</li>
</ul>
<p>Rust 中的自动资源管理可缓解所有这些问题:</p>
<ul>
<li>
<p>资源管理对象可以定义一个析构器(destructor)抽象释放逻辑。通过正常创建一个对象就可以让析构器在正确的时间调用。
对象可以在被使用之后添加析构函数而无需修改客户端代码。</p>
<ul>
<li>注意 Rust 中的析构器不同于带 GC 的语言中的析构器。它们总是在对象不在被使用后执行,并且一定会被执行,除了运行析构器本身不附带任何运行时开销。</li>
</ul>
</li>
<li>
<p>由于资源管理和自动内存管理通过相同的方式工作,这将消除恼人的缩进并且不需要包围额外的代码。</p>
</li>
<li>在 Rust 中,你可以像传递其他类型的对象一样传递资源。如果你将所有权转移到其他作用域,资源将会在新作用域完成时被关闭。
除此之外,借用系统将还会像保证内存一样保证资源不存在“释放后使用”。</li>
</ul>
<p>简而言之,使用同一系统进行内存和资源管理确实有好处。</p>
<p>我不会说 Rust 所有权系统像垃圾回收一样不用耗费任何心力。 但是,Rust 已经做了很多非常聪明的事情来弥补,正如我们所看到的在某些情况下的易用性甚至超过了带垃圾回收机制的语言。</p>
<p>作为交换,你将获得一种非常快速的语言,并且可以绝对安全地直接控制内存。</p>
<p>因此,它开启了一个高级语言用户都可以编写低级代码的时代,这确实让我感到兴奋。同时在社区也可以找到很多人互相学习。</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:fn:1">
<p>当我说“无需”,我的意思是绝大部分都不需要。在带有垃圾回收的语言中,有时你最终还是会直接管理内存, 同样的在 Rust 中你最终还是会直接管理资源。重要的是在两种情况下,主要的编程模型是编程语言替你管理资源。 <a class="footnote-backref" href="#fnref:fn:1" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
</ol>
</div>【译】Rust 借用和生命周期2020-08-26T00:00:00+08:002020-08-26T00:00:00+08:00coldtag:www.linuxzen.com,2020-08-26:/rust-borrow-and-lifetimes.html<blockquote>
<p>译者注:这是我学习 Rust 生命周期对我最有帮助的文章之一,故翻译了一下。</p>
</blockquote>
<p>原文链接:<a href="http://arthurtw.github.io/2014/11/30/rust-borrow-lifetimes.html">Rust Borrow and Lifetimes</a>。</p>
<p>Rust 是一门处于往 1.0 活跃开 …</p><blockquote>
<p>译者注:这是我学习 Rust 生命周期对我最有帮助的文章之一,故翻译了一下。</p>
</blockquote>
<p>原文链接:<a href="http://arthurtw.github.io/2014/11/30/rust-borrow-lifetimes.html">Rust Borrow and Lifetimes</a>。</p>
<p>Rust 是一门处于往 1.0 活跃开发的新语言(译注:1.0 早已发布,目前最新稳定版本 <a href="https://github.com/rust-lang/rust/releases/tag/1.42.0">1.42</a>)。
我必须再写一篇关于我为什么觉得 Rust 很棒的新博客,但是今天我将关注在它的借用(borrow)
和生命周期(lifetimes)系统,这也是常常让包括我在内的 Rust 新手陷入困境的地方。这篇文章假设
你基本了解 Rust,如果还没推荐你先阅读<a href="http://doc.rust-lang.org/guide.html">指南</a>和<a href="http://doc.rust-lang.org/guide-pointers.html">指针指南</a>。</p>
<h2 id="资源所有权和借用">资源所有权和借用</h2>
<p>Rust 通过一个难缠的(sophisticated)借用系统在不用 GC 的情况下达到内存安全。对于任何资源
(栈内存、堆内存、文件句柄等),他们都对应一个唯一的所有者(owner)在需要的情况下处理资源回收。
你可以通过 <code>&</code> 或者 <code>&mut</code> 创建一个新的绑定指向该资源,这被称之为借用或可变借用。编译器确保
所有的所有者(owners)和借用者(borrowers)行为正确。</p>
<h2 id="拷贝和转移-copy-and-move">拷贝和转移(Copy and move)</h2>
<p>在我们开始进入借用系统之前,我们需要知道 Rust 如何处理拷贝和转移。这个 <a href="https://stackoverflow.com/questions/24253344/is-it-possible-to-make-a-type-only-movable-and-not-copyable/24253573#24253573">StackOverflow 答案</a>非常值得一读。
基本上,在赋值和函数调用上:</p>
<ol>
<li>如果值是可拷贝的(copyable)(仅涉及原始(primitive)类型,不涉及如内存或文件句柄的资源),编译器默认进行拷贝。</li>
<li>其他情况,编译器转移(moves)所有权(ownership)并使原来的绑定无效。</li>
</ol>
<p>简而言之,POD(Plan Old Data) => 拷贝,Non-POD(线性类型(linear types))=> 转移。</p>
<p>以下是一些额外的注释供你参考:</p>
<ul>
<li>Rust 拷贝像 C。每一个按值(by-value)使用一个值都是字节拷贝(通过 <code>memcpy</code> 浅拷贝),而不是语义上的拷贝或克隆。</li>
<li>如果想要让一个 POD 结构体变成不可拷贝的,你可以使用一个 <a href="http://doc.rust-lang.org/std/kinds/marker/struct.NoCopy.html">NoCopy</a> 标记,或者实现 <a href="http://doc.rust-lang.org/std/ops/trait.Drop.html">Drop</a> 特性(trait)。</li>
</ul>
<p>转移之后,所有权就转移到了下一个所有者那。</p>
<h2 id="资源回收">资源回收</h2>
<p>Rust 会在任何资源的所有权消失后立刻释放该资源,就这些,当:</p>
<ol>
<li>所有者超出作用域,或</li>
<li>正在持有的所有者改变绑定(原始绑定变成 void)。</li>
</ol>
<h2 id="所有者和借用者的权限-privileges-和限制">所有者和借用者的权限(privileges)和限制</h2>
<p>这一节基于 <a href="http://doc.rust-lang.org/guide.html">Rust Guide</a> 在权限(privileges)一部分提到拷贝和转移。</p>
<p>所有者有一些权限。它可以:</p>
<ol>
<li>控制资源回收。</li>
<li>借出资源,不可变的(可多次借用)或可变的(只能独占),和</li>
<li>交出所有权(通过转移)</li>
</ol>
<p>同时所有者也存在一些限制:</p>
<ol>
<li>
<p>不可变借用期间,所有者不能</p>
<p>a. 改变资源,或者</p>
<p>b. 以可变的方式借出资源。</p>
</li>
<li>
<p>可变借用期间所有者不能</p>
<p>a. 访问该资源,或者</p>
<p>b. 再次借出该资源。</p>
</li>
</ol>
<p>借用者同时也有一些权限。除了访问或者更改借用的资源外,借用者也可以进一步借出(share the borrow):</p>
<ol>
<li>不可变借用者可以借出(拷贝)不可变借用(译注:再次以不可变借用借出)</li>
<li>可变借用者可以交出(转移)可变借用。(可变引用默认使用转移。)</li>
</ol>
<h2 id="代码示例">代码示例</h2>
<p>关于借用我们已经聊的够多了,让我们一起来看一些代码吧(你可以通过 <a href="https://play.rust-lang.org">https://play.rust-lang.org</a> 运行这些 Rust 代码。)
在下面所有的例子中,我们将使用不可拷贝的 <code>struct Foo</code> ,因为它包含了一个装箱(boxed)(堆分配)值。
使用不可拷贝资源可以限制相关操作,让我们更好的学习。</p>
<p>对于每一个代码示例,我们还提供了一个“作用域图表”(scope chart)来展示所有者和借用者的作用域。
图表第一行的大括号和代码中的大括号一一对应。</p>
<h3 id="所有者在可变借用期间不能访问资源">所有者在可变借用期间不能访问资源</h3>
<p>如果我们将代码中的 <code>println!</code> 解除注释,代码将不能编译:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Foo</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">f</span>: <span class="nb">Box</span><span class="o"><</span><span class="n">int</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// mutable borrow</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// error: cannot borrow `a.f` as immutable because `a` is also borrowed as mutable</span>
<span class="w"> </span><span class="c1">// println!("{}", a.f);</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a x * }
owner a |_____|
borrower x |___| x = &mut a
access a.f | error
</code></pre></div>
<p>这违反了所有者限制 #2(a)。如果我们将 <code>let x = &mut a;</code> 在一个嵌套的代码块里:借用
在 <code>println!</code> 之前结束,这段代码将能正常工作:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// mutable borrow</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// mutable borrow ends here</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="fm">println!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">f</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a { x } * }
owner a |_________|
borrower x |_| x = &mut a
access a.f | OK
</code></pre></div>
<h3 id="借用者可以转移可变借用到一个新的借用者">借用者可以转移可变借用到一个新的借用者</h3>
<p>这段代码展示借用者的权限 #2: 可变借用 <code>x</code> 可以将所有权转移可变借用到一个新的借用者 <code>y</code> 。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// mutable borrow</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// move the mutable borrow to new borrower y</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">x</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// error: use of moved value: `x.f`</span>
<span class="w"> </span><span class="c1">// println!("{}", x.f);</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a x y * }
owner a |_______|
borrower x |_| x = &mut a
borrower y |___| y = x
access x.f | error
</code></pre></div>
<p>转移之后,原始的借用者 <code>x</code> 不再能访问借用的资源。</p>
<h2 id="借用作用域-borrow-scope">借用作用域(Borrow scope)</h2>
<p>如果我们开始传递引用( <code>&</code> 和 =&mut=)事情就开始变得有趣,同时也是 Rust 新手们开始困惑的地方。</p>
<h3 id="生命周期-lifetime">生命周期(Lifetime)</h3>
<p>在整个借用过程中,知道借用者的借用什么时候开始和结束非常重要。在<a href="http://doc.rust-lang.org/guide-lifetimes.html">生命周期指南</a>中是这样定义生命周期的:</p>
<blockquote>
<p>A lifetime is a static approximation of the span of execution during which the pointer is valid: it always corresponds to some expression or block within the program.</p>
</blockquote>
<!--quoteend-->
<blockquote>
<p>生命周期是指针有效范围的静态近似值:它始终对应程序中的某些表达式或代码块。</p>
</blockquote>
<p>然而,我更喜欢使用 <strong>借用作用域(borrow scope)</strong> 这个术语去描述借用生效的作用域。请注意它不同于上面生命周期的定义。
(我第一次见到这个术语是在一个 Rust <a href="https://github.com/rust-lang/rfcs/pull/431">RFC 讨论</a> 中,尽管我的定义可能会有所不同。)我会在稍后给出我为什么避免使用生命周期的原因。
现在我们先把生命周期放在一边。</p>
<h3 id="and-borrow">& = borrow</h3>
<p>一些关于借用的事情:</p>
<p>首先,只需要记住 <code>&</code> = 借用, <code>&mut</code> = 可变借用。任何地方你看到一个 <code>&</code> ,那就是一个借用。</p>
<p>其次,当一个 <code>&</code> 出现在任何结构体中(在它的字段中)或者函数/闭包(返回值或者捕获的引用),结构体/函数/闭包就是一个借用者,
并且应用所有的借用规则。</p>
<p>再次,对于每一个借用,都存在一个所有者和一个或多个借用者。</p>
<h3 id="扩展借用作用域">扩展借用作用域</h3>
<p>一些关于借用作用域的事情:</p>
<p>首先,一个借用作用域:</p>
<ul>
<li>是一个借用生效的范围,并且</li>
<li>不一定是借用者的词法作用域,因为借用者可以扩展借用作用域(参见下面)。</li>
</ul>
<p>其次,借用者在赋值或者函数调用中可以通过拷贝(不可变借用)或者转移(可变借用)扩展借用作用域。
接收者(receiver)(可以是新的绑定、结构体、函数或者闭包)变成新的借用者。</p>
<p>再次,借用作用域是所有借用者作用域的并集,并且被借用的资源必须在整个借用作用域里有效。</p>
<h3 id="借用公式">借用公式</h3>
<p>根据最后一点,我们得到一个借用公式:</p>
<blockquote>
<p>资源作用域 >= 借用作用域 = 所有借用者作用域的并集。</p>
</blockquote>
<h3 id="代码示例">代码示例</h3>
<p>让我们看一些扩展作用域的代码示例。结构体 <code>struct Foo</code> 和前面的一样:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="nc">Foo</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kc">false</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// borrow</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="n">a</span><span class="p">;</span>
<span class="w"> </span><span class="c1">// share the borrow with new borrower y, hence extend the borrow scope</span>
<span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">x</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="c1">// error: cannot assign to `a.f` because it is borrowed</span>
<span class="w"> </span><span class="c1">// a.f = box 1;</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a { x y } * }
resource a |___________|
borrower x |___| x = &a
borrower y |_____| y = x
borrow scope |=======|
mutate a.f | error
</code></pre></div>
<p>即使借用发生在 <code>if</code> 代码块之内并且借用者 <code>x</code> 在 <code>if</code> 代码块之后超出作用域,它已经通过赋值 <code>y=x;</code> 扩展了借用作用域,
所以存在两个借用者: <code>x</code> 和 <code>y</code> 。根据借用公式:借用作用域是借用者 <code>x</code> 和借用者 <code>y</code> 作用域的并集:
范围开始第一次借用于 <code>let x = &a;</code> 直到 <code>main</code> 代码块的结尾。(注意绑定 <code>y</code> 在 <code>y=x;</code> 之前不是借用者。)</p>
<p>你可能注意到了由于条件永远是 false <code>if</code> 代码块永远不会执行,但是编译器始终拒绝资源所有者 <code>a</code> 去访问
它的资源。这是因为所有的借用检查发生在编译期,这样程序运行时就不需要做任何事情。</p>
<h2 id="借用多个资源">借用多个资源</h2>
<p>目前为止我们只关注借用单个资源。借用者可以借用多个资源吗?当然!比如一个函数可以接受两个引用然后
基于一些情况返回其中一个,e.g. 其中字段值比较大的那一个。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">max</span><span class="p">(</span><span class="n">x</span>: <span class="kp">&</span><span class="nc">Foo</span><span class="p">,</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="nc">Foo</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="nc">Foo</span>
</code></pre></div>
<p><code>max</code> 函数返回一个 <code>&</code> 指针,因此它是一个借用者。返回的结果可以是输入参数的任意一个,所以它借用了
两鞥额资源。</p>
<h3 id="命名借用作用域-named-borrow-scope">命名借用作用域(Named borrow scope)</h3>
<p>当存在多个 <code>&</code> 指针作为输入,我们需要使用 <strong>命名生命周期(named lifetimes)</strong> 指定它们之间的关系,
参见 <a href="http://doc.rust-lang.org/guide-lifetimes.html#named-lifetimes">Lifetimes Guide</a>。但现在,让我们叫它们 <strong>命名借用作用域(named borrow scopes)</strong> 。</p>
<p>上面的代码没有使用 <strong>命名生命周期</strong> 指定它们之间的关系是不会通过编译器的,i.e. 哪些借用者 <strong>分组(grouped)</strong>
到哪个借用作用域。下面的实现是合法的:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">max</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span><span class="n">x</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">,</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">x</span><span class="p">.</span><span class="n">f</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">y</span><span class="p">.</span><span class="n">f</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="kr">All</span><span class="w"> </span><span class="n">resources</span><span class="w"> </span><span class="kr">and</span><span class="w"> </span><span class="n">borrowers</span><span class="w"> </span><span class="n">are</span><span class="w"> </span><span class="n">grouped</span><span class="w"> </span><span class="kr">in</span><span class="w"> </span><span class="n">borrow</span><span class="w"> </span><span class="n">scope</span><span class="w"> </span><span class="s">'a.)</span>
<span class="s"> max( { } )</span>
<span class="s"> resource *x <--------------></span>
<span class="s"> resource *y <--------------></span>
<span class="s">borrow scope '</span><span class="n">a</span><span class="w"> </span><span class="o"><==============></span>
<span class="w"> </span><span class="n">borrower</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">|</span><span class="n">___</span><span class="o">|</span>
<span class="w"> </span><span class="n">borrower</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">|</span><span class="n">___</span><span class="o">|</span>
<span class="w"> </span><span class="kr">return</span><span class="w"> </span><span class="n">value</span><span class="w"> </span><span class="o">|</span><span class="n">___</span><span class="o">|</span><span class="w"> </span><span class="n">pass</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">caller</span>
</code></pre></div>
<p>在这个函数中,我们有一个借用作用域 <code>'a</code> 和三个借用者:两个输入参数和函数返回结果。
前面提到的借用公式依然生效,但是现在每个被借用的资源必须满足公式。参见下面的例子:</p>
<h3 id="代码示例">代码示例</h3>
<p>在接下来的代码中,我们来使用上面的 <code>max</code> 函数在 <code>a</code> 和 <code>b</code> 之间选择一个更大 <code>Foo</code> :</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="nc">Foo</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kc">false</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">max</span><span class="p">(</span><span class="o">&</span><span class="n">a</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">b</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// error: `b` does not live long enough</span>
<span class="w"> </span><span class="c1">// y = x;</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a { b x ( ) y } }
resource a |________________| pass
resource b |__________| fail
borrow scope |==========|
temp borrower |_| &a
temp borrower |_| &b
borrower x |________| x = max(&a, &b)
borrower y |___| y = x
</code></pre></div>
<p>直到 <code>let x = max(&a, &b)</code> 都一些正常,因为 <code>&a</code> 和 <code>&b</code> 都是尽在表达式中有效的临时引用,
并且第三个借用 <code>x</code> 借用了两个资源(不管最终是 <code>a</code> 或 <code>b</code> ,对于借用检查器而言它都借用了)直到 <code>if</code>
块结束,所以借用作用域是从 <code>let x = max(&a, &b);</code> 到 <code>if</code> 块结尾。两个资源 <code>a</code> 和 <code>b</code> 在整个借用作用域
都有效,因此满足借用公式。</p>
<p>现在如果我们解除最后一个赋值 <code>y = x;</code> 的注释, <code>y</code> 变成第四个借用者,然后借用作用域被扩展到 <code>main</code>
块的结尾,导致资源 <code>b</code> 不能满足公式。</p>
<h2 id="结构体作为借用者">结构体作为借用者</h2>
<p>除了函数和闭包之外,一个结构体也可以通过其字段存储多个引用来借用多个资源。我们通过下面的一些例子
来看看借用公式如何生效的。我们来使用 <code>Link</code> 结构体来保存一个引用(不可变借用):</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Link</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">link</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div>
<h3 id="结构体借用多个资源">结构体借用多个资源</h3>
<p>即使只有一个字段,结构体 <code>Link</code> 也可以借用多个资源:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Link</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">link</span>: <span class="kp">&</span><span class="nc">a</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kc">false</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// error: `b` does not live long enough</span>
<span class="w"> </span><span class="c1">// x.link = &b;</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a x { b * } }
resource a |___________| pass
resource b |___| fail
borrow scope |=========|
borrower x |_________| x.link = &a
borrower x |___| x.link = &b
</code></pre></div>
<p>在上面例子中,借用者 <code>x</code> 从所有者 <code>a</code> 借用资源,借用作用域到 <code>main</code> 块的结尾。So far so good。
如果我们解除最后一个赋值 <code>x.link = &b;</code> 的注释, <code>x</code> 也尝试从所有者 <code>b</code> 借用资源,这会让资源 <code>b</code>
不能满足借用公式。</p>
<h3 id="没有返回值的函数扩展借用作用域">没有返回值的函数扩展借用作用域</h3>
<p>一个没有返回值的函数同样也可以通过输出参数能扩展借用作用域。例如,这个函数 <code>store_foo</code> 接受一个
<code>Link</code> 的可变引用,然后存储一个引用(不可变借用)到 <code>Foo</code> 里:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">store_foo</span><span class="o"><</span><span class="sc">'a'</span><span class="o">></span><span class="p">(</span><span class="n">x</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Link</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">x</span><span class="p">.</span><span class="n">link</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">y</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>在接下来的代码中,被 <code>a</code> 所有的资源是被借用资源; <code>Link</code> 结构体被借用者 <code>x</code> 可变的引用着(i.e. <code>*x</code> 是借用者);
借用作用域直到 <code>main</code> 块的结尾。</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Link</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">link</span>: <span class="kp">&</span><span class="nc">a</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="kc">false</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Foo</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">f</span>: <span class="nc">box</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="c1">// store_foo(x, &b);</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code> { a x { b * } }
resource a |___________| pass
resource b |___| fail
borrow scope |=========|
borrower *x |_________| x.link = &a
borrower *x |___| x.link = &b
</code></pre></div>
<p>如果我们解除最后一个函数调用 <code>store_foo(x, &b);</code> ,这个函数将会尝试将 <code>&b</code> 存储到 <code>x.link</code> ,
将资源 <code>b</code> 作为另外一个被借用的资源,由于 <code>b</code> 的作用域没有覆盖整个借用作用域,导致不满足借用公式。</p>
<h3 id="多个借用作用域">多个借用作用域</h3>
<p>一个函数中可以存在多个借用作用域。例如:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">superstore_foo</span><span class="o"><'</span><span class="na">a</span><span class="p">,</span><span class="w"> </span><span class="o">'</span><span class="na">b</span><span class="o">></span><span class="p">(</span><span class="n">x</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Link</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">,</span>
<span class="w"> </span><span class="n">x2</span>: <span class="kp">&</span><span class="nc">mut</span><span class="w"> </span><span class="n">Link</span><span class="o"><'</span><span class="na">b</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">y2</span>: <span class="kp">&</span><span class="o">'</span><span class="na">b</span> <span class="nc">Foo</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">x</span><span class="p">.</span><span class="n">link</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">y</span><span class="p">;</span>
<span class="w"> </span><span class="n">x2</span><span class="p">.</span><span class="n">link</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">y2</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>这个的函数(可能不是特别有用)中,涉及两个不同的借用作用域。每个借用作用域都有它们自己的作用域公式要满足。</p>
<h2 id="为什么生命周期会造成困惑">为什么生命周期会造成困惑</h2>
<p>最后,我想解释一下为什么我认为 Rust 借用系统使用 <strong>生命周期</strong> 术语会造成困惑(同时避免在这片博文中使用它)。</p>
<p>当我们讨论借用时会涉及到不同类型的“生命周期”:</p>
<p>A. 资源所有者的生命周期(或者 被所有/被借用 资源
B. 被借用的生命周期,i.e. 从开始借用到最后返还
C. 每一个独立的借用者或被借用的指针的生命周期</p>
<p>当有人说“生命周期”,它可以指上面的任何一个。如果涉及多个资源和借用者就会变的更加困惑。
比如,在函数或者结构体生命中一个“命名的生命周期”指哪个?是 A、B 或者 C?</p>
<p>在我们的前一个 <code>max</code> 函数中:</p>
<div class="highlight"><pre><span></span><code><span class="k">fn</span> <span class="nf">max</span><span class="o"><'</span><span class="na">a</span><span class="o">></span><span class="p">(</span><span class="n">x</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">,</span><span class="w"> </span><span class="n">y</span>: <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="o">'</span><span class="na">a</span> <span class="nc">Foo</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">x</span><span class="p">.</span><span class="n">f</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">y</span><span class="p">.</span><span class="n">f</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>生命周期 <code>'a</code> 的意义是什么?它不应该是 A,因为涉及两个资源并且他们有不同的生命周期。也不可能是 C,
因为有三个借用者: <code>x</code> 、 <code>y</code> 和函数的返回值,并且他们也都有不同的生命周期。它是 B 吗?可能。
但是整个借用作用域并不是一个具体的对象,它怎么能有一个“生命周期”呢?称它为生命周期就会造成困惑。</p>
<p>另一种说法是它意味着对被借用资源的最小生命周期要求。一定程度上是有道理的,
但是我们怎么称呼最小生命周期要求“生命周期”?</p>
<p>所有权/借用概念自身已经够复杂了。我会说:对术语“生命周期”的困惑对学习这个概念造成了更多的莫名其妙。</p>
<p>P.S. 使用上面定义的 A、B 和 C,借用公式变成:</p>
<blockquote>
<p>A >= B = C_1,UC_2U...UC_n</p>
</blockquote>
<h2 id="学习-rust-是值得的">学习 Rust 是值得的!</h2>
<p>尽管借用和所有权可能让你花一些时间来掌握(to grok),但是是一个非常有趣的学习。Rust 尝试不用 GC
来实现内存安全,并且目前来看做的非常好。一些人说他们通过学习 Haskell 改变了他们编程的方式。
我认为Rust 同样也值得你学习。</p>
<p>希望这篇博文能提供一些帮助。</p>【译文】理解布隆过滤器2019-05-10T00:00:00+08:002019-05-10T00:00:00+08:00coldtag:www.linuxzen.com,2019-05-10:/understanding-bloom-filter.html<p>原文:<a href="https://osoco.es/thoughts/2019/05/understanding-bloom-filters-with-pharo-smalltalk/">Understanding Bloom filters with Pharo Smalltalk</a>。</p>
<p>本文通过 HTML 转录了 <a href="https://github.com/osoco/PharoPDS">PharoPDS</a> 库及其扩展附带的交互式教程,以探索和理解布隆过滤器。</p>
<p>因此,如果您想在真实 …</p><p>原文:<a href="https://osoco.es/thoughts/2019/05/understanding-bloom-filters-with-pharo-smalltalk/">Understanding Bloom filters with Pharo Smalltalk</a>。</p>
<p>本文通过 HTML 转录了 <a href="https://github.com/osoco/PharoPDS">PharoPDS</a> 库及其扩展附带的交互式教程,以探索和理解布隆过滤器。</p>
<p>因此,如果您想在真实的环境中修改这些数据结构,请尝试在 <a href="https://pharo.org/">Pharo</a> 镜像中<a href="https://github.com/osoco/PharoPDS#install-pharopds">安装这个库</a>,并按照交互式教程并使用提供的自定义工具进行操作。</p>
<h2 id="_1">理解布隆过滤器</h2>
<p>布隆过滤器是一个非常节省空间的数据结构,由 Burton Howard Bloom 于 1970 年所提出(<a href="http://crystal.uta.edu/~mcguigan/cse6350/papers/Bloom.pdf">Space/Time Trade-offs in Hash Coding with Allowable Errors</a>),布隆过滤器用于测试一个元素是否是集合的成员之一。</p>
<p>常规的哈希搜索通过将一系列值存储在哈希表上,不管是基于链表还是开放寻址(<a href="https://en.wikipedia.org/wiki/Open_addressing">open addressing</a>),随着越来越多的元素添加到哈希表中,定位元素的期望时间都会从初始的常量的 O(1) 退化或增加到线性的 O(N)。</p>
<p>布隆过滤器提供了一个替代的数据结构,可以实现在添加元素或检查元素是否是成员上,不管在空间和时间上都可以保证常量级的性能,并且和已经添加到过滤器中的元素数量是无关的。</p>
<p>为了达到如此高效的表现所需要付出的代价是:布隆过滤器是一个概率性的数据结构。Bloom 他在开创性的论文中解释如下:</p>
<blockquote>
<p>新的方法打算比传统相关的方法减少一定量哈希码所需要的空间。通过利用某些应用可以容忍少量的误差来减少空间的使用,特别是那些拥有大量的数据参与无法通过传统的方法保留核心哈希区域的应用。</p>
</blockquote>
<h2 id="_2">背景</h2>
<p>Donald Knuth 在他著名的 <a href="https://www-cs-faculty.stanford.edu/~knuth/taocp.html">The Art of Computer Programming</a> 中写下了如下对布隆过滤器的描述:</p>
<blockquote>
<p>想象一个拥有非常大量的数据且如果搜索没有成功则无需完成任何计算的搜索应用。比如,我们可能想检查一些人的信用评级或者护照编号,如果文档中没有该人的任何记录我们就无需做任何调查。类似地,在计算机排版应用程序中,我们可能有一个简单算法可以正确地连接大多数单词,但该算法会对大约 5W 个异常单词无效; 如果我们在异常文件中找不到该单词,就可以放心的使用该简单算法。</p>
</blockquote>
<p>同时 Andrei Broder 和 Michael Mitzenmacher 在著名的布隆过滤器原理(The Bloom Filter Principle(<a href="https://www.eecs.harvard.edu/~michaelm/postscripts/im2005b.pdf">Internet Mathematics Vol. 1, No. 4: 485-509Network Applications ofBloom Filters: A Survey</a>))中提出:</p>
<blockquote>
<p>在一个空间宝贵的情况下要使用列表或集合并且可以接受误报,则可以考虑使用布隆过滤器。</p>
</blockquote>
<h2 id="_3">真实世界的例子</h2>
<ul>
<li>Medium 使用布隆过滤器避免推荐给用户已经读过的文章。</li>
<li>Google Chrome 使用布隆过滤器识别恶意 URL。</li>
<li>Google BigTable,Apache HBbase 和 Apache Cassandra 使用布隆过滤器减少对不存在的行和列的查找。</li>
<li>Squid Web 代理使用布隆过滤器处理缓存摘要。</li>
</ul>
<h2 id="_4">基本理解</h2>
<p>布隆过滤器可以通过舍弃元素的特征来高效的存储大规模集合,比如它仅将通过算法对每一个元素应用哈希函数得到的数字存储到一系列位上进行关联。</p>
<p>事实上,布隆过滤器通过一个长度(m)和不同哈希函数的数量(k)的比特数组(bit arrary)来表示。</p>
<p>该数据结构仅支持两种操作:</p>
<ul>
<li>添加一个元素到集合中来,</li>
<li>测试一个元素是否集合的成员。</li>
</ul>
<p>布隆过滤器的数据结构是一个初始比特位都为 0 的比特数组,代表布隆过滤器为空。</p>
<p>举个例子,考虑如下示例中创建的布隆过滤器表示一个包含 10 个元素的集合并且误报率(FPP, False Positive Probability)为 <code>0.1</code> (10%)。</p>
<p>一个空的布隆过滤器将通过一个长度为 <code>m=48</code> (storageSize) 和 4 个哈希函数的比特数组用以支持生产范围在 <code>{1, 2, ..., m}</code> 的值。表示如下:</p>
<div class="highlight"><pre><span></span><code><span class="nf">emptyBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="nc">PDSBloomFilter</span> <span class="nf">new:</span> <span class="m">10</span> <span class="nf">fpp:</span> <span class="m">0.1</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">eqauls:</span> <span class="m">0</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">hashes</span> <span class="nf">equals:</span> <span class="m">4</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bllom</span> <span class="nf">storageSize</span> <span class="nf">equals:</span> <span class="m">48</span><span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/empty-bloom-filter.png"></p>
<p>要插入一个元素 <code>x</code> 到布隆过滤器中,需要对元素 <code>x</code> 应用到每一个哈希函数 $h_i$ 上并计算它的值为 $j=h_i(x)$ ,然后将布隆过滤器中 <code>j</code> 对应的位设置为 1 。</p>
<p>作为一个例子,我们将在上面的过滤器中插入一些城市的名字。让我们通过 <code>'Madrid'</code> 开始:</p>
<div class="highlight"><pre><span></span><code><span class="nf">withMadridBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">emptyBloomfilter</span><span class="p">.</span>
<span class="nv">bloom</span> <span class="nf">add:</span> <span class="s">'Madrid'</span> <span class="nf">asByteArray</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">1</span><span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/withMadrid-bloom-filter.png"></p>
<p>布隆过滤器计算出了 4 个哈希值用以在集合中找到关联 <code>'Madrid'</code> 的比特位。如上图所展示,布隆过滤器设置了 9, 18, 39 和 48 。</p>
<p>不同的元素可能共享一个比特位,比如现在我们添加另一个城市 <code>'Barcelona'</code> 到上面相同的布隆过滤器中:</p>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/withMadridAndBarcelona-bloom-filter.png"></p>
<p>如你所见,在添加 <code>'Barcelona'</code> 到上面布隆过滤器之后只有 30, 36 和 42 所对应的比特位被设置,也就意味着元素 <code>'Madrid'</code> 和 <code>'Barcelona'</code> 共享了一个比特位。</p>
<p>要想测试给定的元素 <code>x</code> 是否在布隆过滤器之中,只需要检查所有的哈希函数 <code>k</code> 计算出的对应的比特位。如果所有位都被设置来,则表示元素 <code>x</code> 可能在布隆过滤器中,否则元素 <code>x</code> 一定不在其中。</p>
<p>元素存在不确定性是由于一些比特位可能是被之前添加的其他不同的元素所设置。</p>
<p>考虑前面的例子,在已经添加了元素 <code>'Madrid'</code> 和 <code>'Barcelona'</code> 的情况下,我们来测试元素 <code>'Barcelona'</code> 是否是该过滤器的成员,布隆过滤器计算出该元素的 4 个哈希值并且检查对应的比特位是否被设置,结果显示字符串 <code>'Barcelona'</code> 可能存在于过滤器之中,然后 <code>contains: 'Barcelona'</code> 返回 <code>true</code> :</p>
<div class="highlight"><pre><span></span><code><span class="nf">withMadridAndBarcelonaCheckBarcelonaBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">withMadridBloomFilter</span><span class="p">.</span>
<span class="nv">bloom</span> <span class="nf">add:</span> <span class="s">'Barcelona'</span> <span class="nf">asByteArray</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">2</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> (<span class="nv">bloom</span> <span class="nf">contains:</span> <span class="s">'Barcelona'</span> <span class="nf">asByteArray</span>)<span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p>现在如果我们检查元素 <code>'Berlin'</code> ,布隆过滤器为了找到对应的比特位会通过如下方式计算它的哈希值:</p>
<div class="highlight"><pre><span></span><code><span class="nf">withBerlinBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">emptyBloomFilter</span><span class="p">.</span>
<span class="nv">bloom</span> <span class="nf">add:</span> <span class="s">'Berlin'</span> <span class="nf">asByteArray</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">1</span><span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/withBerlin-bloom-filter.png"></p>
<p>我们看到比特位 27 和 33 没有被设置,所以元素 <code>'Berlin'</code> 一定不存在于布隆过滤器中,所以 <code>contains:</code> 方法返回 <code>false</code> :</p>
<div class="highlight"><pre><span></span><code><span class="nf">withMadridAndBarcelonaCheckBerlinBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">withMadridBloomFilter</span><span class="p">.</span>
<span class="nv">bloom</span> <span class="nf">add:</span> <span class="s">'Barcelona'</span> <span class="nf">asByteArray</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">2</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> (<span class="nv">bloom</span> <span class="nf">contains:</span> <span class="s">'Berlin'</span> <span class="nf">asByteArray</span>) <span class="nf">not</span><span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p>布隆过滤器的结果也可能是错误的。例如,考虑元素 <code>'Roma'</code> 的 4 个哈希值 36 由于碰撞已经在我们的示例过滤器中设置,所以 <code>contains:</code> 方法认为该元素可能已经存在于过滤器中。</p>
<div class="highlight"><pre><span></span><code><span class="nf">withMadridAndBarcelonaCheckRomaBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">withMadridBloomFilter</span><span class="p">.</span>
<span class="nv">bloom</span> <span class="nf">add:</span> <span class="s">'Barcelona'</span> <span class="nf">asByteArray</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">2</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> (<span class="nv">bloom</span> <span class="nf">contains:</span> <span class="s">'Roma'</span> <span class="nf">asByteArray</span>)<span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p>就像我们所了解到的,我们并没有添加过该元素到布隆过滤器中,所以这是一个误报的例子。在这个特别的案例中,比特位 <code>36</code> 已经被前面的元素 <code>'Barcelona'</code> 所设置。</p>
<h2 id="_5">特性</h2>
<h3 id="_6">误报</h3>
<p>如我们前面所展示,布隆过滤器会对一些不是集合中的成员的元素返回 <code>true</code> 。这种情况被成为误报(false positive),产生于哈希碰撞导致的巧合而将不同的元素存储在相同的比特位上。在测试操作中无法得知我们对比的特定的位是否被相同的哈希函数所设置。</p>
<p>幸运的是,布隆过滤器有一个可预测的误报率(FPP):</p>
<p>$$P_fp\approx\left(1-e^{-\frac{kn}{m}}\right)^k$$</p>
<ul>
<li><code>n</code> 是已经添加元素的数量;</li>
<li><code>k</code> 哈希的次数;</li>
<li><code>m</code> 布隆过滤器的长度(如比特数组的大小)。</li>
</ul>
<p>极端情况下,当布隆过滤器没有空闲空间时(满),每一次查询都会返回 <code>true</code> 。这也就意味着 <code>m</code> 的选择取决于期望预计添加元素的数量 <code>n</code> ,并且 <code>m</code> 需要远远大于 <code>n</code> 。</p>
<p>实际情况中,布隆过滤器的长度 <code>m</code> 可以根据给定的误报率(FFP)的和期望添加的元素个数 <code>n</code> 的通过如下公式计算:</p>
<p>$$m=-\frac{n\ln{P_{fp}}}{(\ln2)^2}$$</p>
<p>对于 $\frac{m}{n}$ 比率表示每一个元素需要分配的比特位的数量,也就是哈希函数 <code>k</code> 的数量可以调整误报率。通过如下公式来选择最佳的 <code>k</code> 可以减少误报率(FPP):</p>
<p>$$k=\frac{m}{n}\ln2$$</p>
<p>我们实现的 <code>PDSBloomFilter</code> 建立在指定预计添加元素的数量( <code>n</code> )和误报率( <code>FPP</code> )之上,然后使用上面的公式去计算最优的哈希次数和比特数组的长度。</p>
<p>比如,要处理 10 亿个元素并且保持 2% 左右的误报率我们需要如下布隆过滤器:</p>
<div class="highlight"><pre><span></span><code><span class="nf">oneBillionBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="nc">PDSBloomFilter</span> <span class="nf">new:</span> <span class="m">1000000000</span> <span class="nf">fpp:</span> <span class="m">0.02</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">size</span> <span class="nf">equals:</span> <span class="m">0</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">hashes</span> <span class="nf">equals:</span> <span class="m">6</span><span class="p">.</span>
<span class="bp">self</span> <span class="nf">assert:</span> <span class="nv">bloom</span> <span class="nf">storageSize</span> <span class="nf">equals:</span> <span class="m">8142363337</span><span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p>如你所见,最优的哈希次数是 6 并且过滤器的长度是 $8.14 x 10^9$ 个比特位,大约占用 1 GB 内存。</p>
<h3 id="_7">非误报</h3>
<p>如果布隆过滤器返回特定的元素不是成员之一,那么该元素就绝对不在集合之中。</p>
<h3 id="_8">无法删除</h3>
<p>要想从过滤器中删除一个元素需要在比特数组中取消设置相应数量( <code>k</code> )的比特位。不幸的是,由于哈希碰撞导致多个元素会共享比特位导致了一个比特位可能关联了多个元素。</p>
<h2 id="_9">分析</h2>
<p>前面计算误报率(FPP)的公式为了容易计算从而假设了哈希函数 <code>k</code> 的随机是均匀分布的。</p>
<p>换句话说,一旦将期望的 <code>n</code> 个元素添加到数据结构中, <code>fpp</code> 在 <code>PDSBloomFilter</code> 初始化时指定的值应该解释为期望的 <code>fpp</code> 的值。</p>
<p>例如,对于一个刚刚创建依然保持为空的布隆过滤器的 <code>fpp</code> 值为 0,如下图所见:</p>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/empty-filter-fpp-curve.png"></p>
<p><code>fpp</code> 的值会随着不断的向过滤器中添加元素而增长,并随着布隆过滤器中的元素数量达到期望值而最终达到目标 <code>fpp</code> ,我们通过如下填充布隆过滤器:</p>
<div class="highlight"><pre><span></span><code><span class="nf">fullBloomFilter</span>
<span class="err"><</span><span class="nv">gtExample</span><span class="nf">></span>
<span class="o">|</span><span class="nv"> bloom </span><span class="o">|</span>
<span class="nv">bloom</span> <span class="o">:=</span> <span class="bp">self</span> <span class="nf">emptyBloomFilter</span><span class="p">.</span>
<span class="m">1</span> <span class="nf">to:</span> <span class="nv">bloom</span> <span class="nf">targetElements</span>
<span class="nf">do:</span> [ <span class="o">:</span><span class="nv">each</span> <span class="o">|</span> <span class="nv">bloom</span> <span class="nf">add:</span> <span class="nv">each</span> <span class="nf">asString</span> <span class="nf">asByteArray</span> ]<span class="p">.</span>
<span class="o">^</span> <span class="nv">bloom</span>
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/full-filter-fpp-curve.png"></p>
<p>不过,你应该知道上面的 FPP 曲线是一个理论值并且实际观测到的 FPP 将会取决于特定的数据集和所使用的哈希函数。为了实验检查我们的实现的优点,我们进行了如下分析:</p>
<ol>
<li>随机生成一个包含邮件地址的列表并插入到布隆过滤器。</li>
<li>随机生成一个包含邮件地址的列表不插入到该过滤器中。</li>
<li>统计在该过滤器中搜索缺失地址的误报情况。</li>
</ol>
<p>我们运行的实验值在该过滤器期望元素个数的 10 到 1.5 倍(步长 10)。对于每种实验的元素数量运行 10 次。该分析通过一个图像展示了该过滤器的如下指标:</p>
<ul>
<li>理论的 FPP 曲线(蓝色)。</li>
<li>每次实验测量到的平均 FPP 值(灰色十字)。</li>
<li>实际的 FPP 曲线(红色)</li>
<li>和标准差(红色阴影)</li>
</ul>
<p>例如,如下代码运行上面的实验分析并且通过一个图展示一个包含 100 个元素和 3% 的 FPP 的布隆过滤器结果:</p>
<div class="highlight"><pre><span></span><code><span class="nc">PDSBloomFilterAnalysis</span> <span class="nf">openFor:</span> (<span class="nc">PDSBloomFilter</span> <span class="nf">new:</span> <span class="m">100</span> <span class="nf">fpp:</span> <span class="m">0.03</span>)
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/analysis-fpp.png"></p>
<h2 id="_10">压测</h2>
<p>一个布隆过滤器每次操作只需要固定次数的探测( <code>k</code> ),所以在每次插入和查找的处理只需要 <code>O(k)</code> 的时间复杂度,也就是常量级的。</p>
<p>例如,如下代码将基于在前面被填充了 10 个元素的布隆过滤器上执行查找操作来运行一个微型压测并且展示每秒钟运行查找操作的次数:</p>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/bloom-benchmark-little.png"></p>
<p>由于时间复杂度是常量级的,那么一个前面添加了 100W 个元素的布隆过滤器的查找结果应该和上面类似:
<img alt="" src="/static/upload/trans-understanding-bloom-filter/bloom-benchmark-million.png"></p>
<p>一个基于 <code>Collection</code> 数据结构的简单实现将是线性的 <code>O(n)</code> 的时间复杂度,一个有序集合的最优情况将会是 <code>O(log n)</code> 的时间复杂度。你可以通过观察下面压测使用一个长度为 <code>n</code> 的 <code>OrderedCollection</code> 集合的降级行为来检验:</p>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/collection-benchmark-little.png"></p>
<p>VS</p>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/collection-benchmark-million.png"></p>
<h2 id="_11">试玩</h2>
<p>PharoPDS 提供了一个简单的工具让你可以探索和尝试布隆过滤器。</p>
<p><code>PDSBloomFilterPlayground</code> 允许你创建一个布隆过滤器尝试一些操作并且进行可视化。甚至你可以基于 UI 运行压测和调优。</p>
<p>可以通过如下代码运行:</p>
<div class="highlight"><pre><span></span><code><span class="nc">PDSBloomFilterPlayground</span> <span class="nf">open</span>
</code></pre></div>
<p><img alt="" src="/static/upload/trans-understanding-bloom-filter/bloom-filter-playground.png"></p>
<h2 id="_12">引用</h2>
<ul>
<li>
<p>On Probabilistic Data Structures:</p>
<ul>
<li>Burton H. Bloom. <a href="http://crystal.uta.edu/~mcguigan/cse6350/papers/Bloom.pdf">Space/Time Trace-offs in Hash Coding with Allowable Errors</a> (1970).</li>
<li>Donald E. Knuth. <a href="https://www-cs-faculty.stanford.edu/~knuth/taocp.html">The Art of Computer Programming</a> Volume 3: Sorting and Searching.</li>
<li><a href="http://llimllib.github.io/bloomfilter-tutorial/">Bloom Filters by Example</a>.</li>
<li>Andrii Gakhov, <a href="https://www.gakhov.com/books/pdsa.html">Probabilistic Data Structures and Algorithms for Big Data Applications</a>.</li>
</ul>
</li>
<li>
<p>On Moldable Development:</p>
<ul>
<li>Andrei Chis’s PhD, “Moldable Tools” (2016). Available at <a href="http://scg.unibe.ch/archive/phd/chis-phd.pdf">http://scg.unibe.ch/archive/phd/chis-phd.pdf</a>.</li>
<li>Andrei Chis, “Playing with Face Detection in Pharo” (2018). Available at <a href="https://medium.com/@Chis_Andrei/playing-with-face-detection-in-pharo-e6dd297e0ca3">https://medium.com/@Chis_Andrei/playing-with-face-detection-in-pharo-e6dd297e0ca3</a>.</li>
<li>Tudor Girba, “Moldable Development” (2018). Available at <a href="https://www.youtube.com/watch?v=IcwHaF5aRTM">https://www.youtube.com/watch?v=IcwHaF5aRTM</a>.</li>
</ul>
</li>
</ul>【译文】什么是幂等2019-05-05T00:00:00+08:002019-05-05T00:00:00+08:00coldtag:www.linuxzen.com,2019-05-05:/what-is-idempotence.html<p>原文:<a href="https://lispcast.com/what-is-idempotence/">WHAT IS IDEMPOTENCE</a></p>
<p>(原文是一个视频的文字记录版,有兴趣的可以看原文和原文中的视频,本文只翻译文字并结合自己的一些理 …</p><p>原文:<a href="https://lispcast.com/what-is-idempotence/">WHAT IS IDEMPOTENCE</a></p>
<p>(原文是一个视频的文字记录版,有兴趣的可以看原文和原文中的视频,本文只翻译文字并结合自己的一些理解做一些整理。)</p>
<h3 id="_1">引子</h3>
<p>幂等意味着可以重复,也就是说你可以安全的重试一个操作不会产生任何问题。经典的例子是电梯按钮:你按两次并不会叫来两辆电梯。同时我们来探索为什么在一个 Email 服务中需要这个特性。</p>
<p>什么是幂等?为什么幂等对于分布式编程非常有用?通过这篇文章,你将知道如何在你自己的系统中实现幂等。</p>
<p>幂等之所以重要是因为它抓住了安全重试的本质。没有安全重试也就没有办法实现一个安全的分布式协议。</p>
<h3 id="_2">什么是幂等?</h3>
<p>幂等的本质就是你可以请求两次(或多次),但是和请求一次的所产生的副作用一致。经典的例子是电梯按钮。假设你来到一组电梯面前按下按钮,按钮亮起并呼叫电梯。然后另一个人同样来到这组电梯面前,按钮已经点亮但他依然按下相同的按钮。</p>
<p>我们都知道这样做不会产生任何效果(副作用),但是由于某些原因我们依然想这么做,仅仅为了以防万一。也许他是对的,也许一开始信号并没有传递给电梯。由于这样做没有任何坏处,所以为什么不呢?这就是我们想在我们的分布式系统中贯彻的理念。</p>
<p>技术上讲,这是一个属于代数的概念。当我们讨论按下按钮时,我们对现实的世界产生了一个有效的副作用。而在代数中,幂等是纯函数和数学函数的属性。幂等意味着你将字符串中的字母转换成大写两次对其结果没有任何影响,因为第一次操作就可以了。技术上来说,如果将函数 $F$ 应用到一个值上,如 $F(x)$ ,与将 $F(x)$ 应用到 $F$ 是相同的( $F(x)=F(F(x))$ )。</p>
<p>我们应用了两次函数 $F$ 和应用一次所产生的副作用是一致的,也就是说重复无关。我按下按钮,第二次按下不会产生任何副作用和问题。如果我应用两次函数,第二次不会产生任何副作用。第一次有副作用,那么接下来第二次、第三次、第四次等等都不会产生副作用。</p>
<h3 id="_3">为什么幂等很重要?</h3>
<p>在一个分布式系统,特别是分布式系统,我们会面临消息通过一个不可靠的网络传输。基本上,如果你发送了一个消息,消息可能没有送达并且你无法感知。也就是说你无法确定消息是否到达。</p>
<p>有时,你得到连接中断的消息,你会知道消息没有到达。但有时超时导致你无法获得反馈。是消息已经到达但是由于 ACK 超时没有收到反馈,还是消息压根就没有到达?其他系统崩溃?系统崩溃发生在消息发送之前还是之后?由于系统已经崩溃,你无法得知。</p>
<p>Email 实际上十一个很好的例子,因为同一封邮件你不想发送两次。让我们假设我们有一个邮件服务并且通过它发送一个消息:“请将这封邮件发送给我的客户”。你没有得到任何反馈,你将会怎么做?发生了什么?你会尝试重新发送吗?如果邮件已经发送了呢?再发一次相同的邮件?如果不重新发送客户将无法收到收件。</p>
<p>这是一个真实存在的商业问题。幂等将可以解决此问题:如果我重新发送邮件,但不会破坏任何事情(不会产生第二次副作用)。就像前面说的电梯按钮一样,我可以一直重复发送这封邮件。我可以重复发送一百次,但这封邮件只会被投递一次。</p>
<p>幂等实现了请求次数和实际产生副作用的次数的解偶。我可以请求一百次,但是只产生一次副作用。这是有些时候你真正想实现的:通过限制的消息能够进行安全的重试。</p>
<p>“我不知道这是否行得通,但是我将会再试一次。”这将是你系统中非常好的属性。</p>
<h3 id="_4">如何实现幂等?</h3>
<p>假设基于邮件服务,最简单的方法是需要通过一些方法去识别一封邮件的唯一性(ID)。比如:“这是这封邮件的 ID,如果你使用同一个 ID 发送同一封邮件多次,仅投递一次。”</p>
<p>邮件服务为了实现完备的幂等性将会记住所有发送过的邮件 ID。但是通常这是不切实际的。因为当数量达到上百万时你无法记住每一个 ID。</p>
<p>由于可能会追溯到许多年以前,并且一个请求不可能需要以年记才能到达。所以实际上,你可能需要一个窗口,如:“我们只保留 3 天的 ID”。也就是说你可以通过相同的 ID 重复的发送三天以内的邮件,并且不会产生第二次投递。在实际场景中,你需要平衡内存的需求和你系统所实现的安全重试机制来探索实际的限制。</p>
<p>注意,唯一性的识别(ID)非常重要。如果没有唯一性识别(ID)的概念,那么怎么实现同一个消息再次发送?如果想给一个人发送两封邮件,我需要能发送两封邮件给他。我需要一种方法标志两封邮件不相同。如果我想重试,我需要一种方法标志这封邮件和那封邮件是同一个。</p>
<p>你的请求需要一些 ID。参照电梯按钮,可能 ID 就在电梯服务的电子系统内部,它知道我按的哪个按钮:是三楼向上还是四楼向下。按钮的 ID 可以首先让按钮亮起,并且保持到不再需要后再关闭。</p>
<p>ID 可能用在多个地方。用在一个请求上:“我们需要电梯向上到三层因为我们知道那个按钮和它的意义”。也可以是:“我已经发送过了到三层电梯的信号,我不需要再次发送。”。</p>
<p>一旦确定了 ID,你可以使用一个已经是幂等的数据结构的操作,比如集合(set)。如果你用一个数字集合存储每个邮件的唯一数字 ID。当邮件服务发送完邮件,通过将数字 ID 添加到集合用以记录。如果你添加到该集合两次,你实际上已经获得到幂等性。</p>
<p>同电梯一样,如果你有一个按钮包含来字符串 ID,比如 <code>third-floor-up</code> 、 <code>third-floor-down</code> 、 <code>fourth-floor-up</code> 、 <code>fourth-floor-down</code> , 通过将这些 ID 存放到集合内表示对应的按钮被请求而激活。这也就意味着你可以按下多次,第二次将不会产生任何效果(副作用)。</p>
<p>当然,现在没有考虑按下按钮将电梯送到某一层的实际的实现。但是和邮件相同,现在的确考虑了发送还是不发送邮件。</p>
<p>要想得知你是否想要发送它其实很简单。在添加某一项 ID 到集合中之前,询问集合:“你是否包含这个 ID ?”。如果已经包含则结束操作,如果不包含则发送邮件并将 ID 添加到集合内。除了集合还有一些其他的数据结构是幂等的,比如哈希表。</p>
<p>前面提到的将字符串中的字母转换为大写,这是一种幂等的操作。</p>
<h3 id="_5">总结</h3>
<p>让我们来概括一下。幂等意味着重复无关。他是一种代数中函数和操作的属性,但是我们将之扩展到现实世界中的一些行为。在分布式操作系统中我们之所以需要它因为我们需要在分布式系统中安全的重试。幂等让我们从请求次数和请求完成解偶出来。你可以很容易通过一些数据结构和对应的操作来实现。为了实现幂等需要每一个消息都有一个 ID。</p>
<hr>
<p>忽略以下和主题无关的翻译:</p>
<p>Do yourselves a favor and look for some services that need to happen exactly once. Could be something like sending an email. Could be writing a message to a log. Could be some user setting in your user-panel, and wrap them in something like a data structure that makes them idempotent.</p>
<p>Do me a favor please and share this with friends. If you found it valuable, they might find it valuable, too. Also, if you found it valuable, you probably want to subscribe. That way you’ll get all of the other new episodes as they come out. You won’t miss that value that you have already discovered.</p>
<p>I like to be in deep discussions with smart people. Please email me. I’m eric@lispcast.com or get in a discussion on Twitter. I try to use Twitter as a discussion medium. I’m @ericnormand with a D there.</p>
<p>Also, you can find me on LinkedIn. I’m trying to get better at LinkedIn. It’s a little hard for me. If that’s where you like to connect, let’s connect and start having a conversation.</p>
<p>All right. See you later.</p>Python 3.8 新增 multiprocessing.SharedMemory 支持共享内存2019-02-28T00:00:00+08:002019-02-28T00:00:00+08:00coldtag:www.linuxzen.com,2019-02-28:/python-38-shared-memory.html<p>Python 在 2019-02-25 释出了 3.8 早期预览版 <a href="https://www.python.org/downloads/release/python-380a2/">3.8.0a2</a>,其中新增了 <a href="https://docs.python.org/3.8/library/multiprocessing.shared_memory.html">multiprocessing.SharedMemory</a> 用以支持共享内存,大大提高多进程之间通信效率。简单看了一下 …</p><p>Python 在 2019-02-25 释出了 3.8 早期预览版 <a href="https://www.python.org/downloads/release/python-380a2/">3.8.0a2</a>,其中新增了 <a href="https://docs.python.org/3.8/library/multiprocessing.shared_memory.html">multiprocessing.SharedMemory</a> 用以支持共享内存,大大提高多进程之间通信效率。简单看了一下实现代码主要涉及如下 Python 模块</p>
<ul>
<li>内置类型 <a href="https://docs.python.org/3.8/library/stdtypes.html?highlight=memoryview#memoryview">memoryview</a></li>
<li><a href="https://docs.python.org/3.8/library/mmap.html">mmap</a></li>
</ul>
<p>在 POSIX 平台下共享内存创建过程如下:</p>
<ol>
<li>基于 <code>tmpfs</code> 打开或创建具名(文件名)的共享内存,得到文件描述符</li>
<li>通过 <code>mmap</code> 将文件描述符映射进程的内存地址空间</li>
<li>通过 <code>memoryview</code> 直接访问经过 <code>mmap</code> 映射后的的内存地址空间</li>
</ol>
<h2 id="_1">锁的问题</h2>
<p><code>memoryview</code> 通过如下方式使用:</p>
<div class="highlight"><pre><span></span><code><span class="n">s</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="sa">b</span><span class="s1">'aaa'</span><span class="p">)</span>
<span class="n">m</span> <span class="o">=</span> <span class="nb">memoryview</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="n">m</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">98</span>
<span class="nb">print</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="c1"># outputs: bytearray(b'baa')</span>
</code></pre></div>
<p>当上面代码执行 <code>m[0] = 98</code> 时实际上调用的是 C 代码 <a href="https://sourcegraph.com/github.com/python/cpython@23f4589b4b7c4a51950a87175ce7fb31b89c8532/-/blob/Objects/memoryobject.c#L2455:1">memory_ass_sub</a>,然后调用 <a href="https://sourcegraph.com/github.com/python/cpython@23f4589b4b7c4a51950a87175ce7fb31b89c8532/-/blob/Objects/memoryobject.c#L1743:9">PACK_SINGLE</a> 通过 <code>memcpy</code> 覆盖指针原有的值。</p>
<p>所以直接操作 <code>multiprocessing.SharedMemory</code> 会产生数据竞争,不应该直接使用,应该使用 <a href="https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Value">multiprocessing.Value</a> 和 <a href="https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Array">multiprocessing.Array</a> 这种更高层的抽象,锁在这一层级实现。</p>
<h2 id="_2">参见</h2>
<p>更多关于共享内存参见:</p>
<ul>
<li><a href="http://hustcat.github.io/shared-memory-tmpfs/">浅谈 Linux 共享内存</a></li>
<li><a href="http://man7.org/training/download/posix_shm_slides.pdf">POSIX Shared Memory</a></li>
<li><a href="https://www.ibm.com/developerworks/cn/linux/l-ipc/part5/index1.html">共享内存(上)</a></li>
<li><a href="https://www.ibm.com/developerworks/cn/linux/l-ipc/part5/index2.html">共享内存(下)</a></li>
</ul>
<hr>译文:Go 内存分配器可视化指南2019-02-23T00:00:00+08:002019-02-23T00:00:00+08:00coldtag:www.linuxzen.com,2019-02-23:/go-memory-allocator-visual-guide.html<p>当我第一次开始尝试理解 Go 语言的内存分配器时,整个过程让我抓狂。一切看起来都像一个神秘的黑盒子。因为几乎所有 …</p><p>当我第一次开始尝试理解 Go 语言的内存分配器时,整个过程让我抓狂。一切看起来都像一个神秘的黑盒子。因为几乎所有技术魔法(technical wizardry)都隐藏在抽象之下,所以你需要一层一层的剥离才能去理解它。</p>
<p>我们将通过这篇文章来一层层的剥离这些细节。如果你想学习所有关于 Go 内存分配器的知识,那么这篇文章正适合你。</p>
<h2 id="_1">物理内存和虚拟内存</h2>
<p>每一个内存分配器都需要运行在由底层操作系统管理的虚拟内存空间(Virtual Memory Space)之上。</p>
<p>下图是一个物理内存单元(Physical Memory Cell)的简要说明(非精准)</p>
<p><img alt="A simple illustration of a Physical Memory Cell" src="static/upload/go-memory-allocator-visual-guide/5c6f9646b9048569f7000001.png"></p>
<p>一个内存单元的概述经过大大简化之后描述如下:</p>
<ol>
<li>地址线(Address line)(晶体管做的开关)用于访问电容器(数据到数据线(Data Lines))。</li>
<li>如果地址线有电流流动(显式为红色),数据线可以写入到电容器,所以电容器带电,逻辑值表示 “1”。</li>
<li>如果地址线没有电流流动(显式为绿色),数据线不可以写入到电容器,所以电容器不带电,逻辑值表示 “0”</li>
<li>当 CPU 需要从 RAM 中“读取”值,则顺着“地址线(ADDRESS LINE)”(关闭开关)发送一个电流。如果电容器带电,则电流流向“数据线(DATA LINE)”(值为 1);否则没有电流流向数据线,所以电容器保持不带电(值为 0)。</li>
</ol>
<p>下图简单的描述 CPU 和物理内存单元如何交互</p>
<p><img alt="Simple Illustration of how a Physical Memory Cell interacts with CPU" src="static/upload/go-memory-allocator-visual-guide/5c6fa15bb9048569f7000002.png"></p>
<p><strong>数据总线(Data Bus)</strong>:用于在 CPU 和内存中间传输数据。</p>
<p>还有一点关于<strong>地址线(Address line)</strong>和<strong>按字节寻址(Addressable bytes)</strong>。</p>
<p>下图是 CPU 和物理内存之间地址线的说明</p>
<p><img alt="Illustrative Representation of an Address Line between CPU and Physical Memory." src="static/upload/go-memory-allocator-visual-guide/5c6fa25eb9048569f7000003.png"></p>
<ol>
<li><a href="https://zh.wikipedia.org/zh/%E5%8A%A8%E6%80%81%E9%9A%8F%E6%9C%BA%E5%AD%98%E5%8F%96%E5%AD%98%E5%82%A8%E5%99%A8">DRAM</a> 中的每一个字节都分配了一个唯一的数字标识符(地址)。“<strong>物理字节 != 地址线的数量(Physical bytes present != Number of address line)</strong>”(e.g. 16 位 Intel 8088、<a href="https://zh.wikipedia.org/wiki/%E7%89%A9%E7%90%86%E5%9C%B0%E5%9D%80%E6%89%A9%E5%B1%95">PAE</a>)</li>
<li>每一个“地址线”可以发送 1-bit 的值,用于表示给定字节地址中的“一个位(SINGLE BIT)”</li>
<li>
<p>在我们的上面给出的图中,我们有 32 个地址线。所以每个 <strong>字节(BYTE)</strong> 都有“32 位”作为地址。</p>
<ul>
<li><code>[ 00000000000000000000000000000000 ]</code> — 低内存地址</li>
<li><code>[ 11111111111111111111111111111111 ]</code> — 高内存地址</li>
</ul>
</li>
<li>
<p>由于我们每字节都有一个 32 位的地址,所以我们的地址空间包含 2 的 32 次方个可寻址字节(bytes)(4GB)。</p>
</li>
</ol>
<p>综上所述,可寻址的字节数量取决于地址总线的数量,所以对于 64 个地址线最大可寻址 2 的 64 次方个字节数(16 <a href="https://zh.wikipedia.org/wiki/%E8%89%BE%E5%AD%97%E8%8A%82">EB</a>),但是由于大部分架构实际上仅使用 48-bit 地址线(AMD)和 42-bit 地址线(Intel)作为 64-bit 指针,所以理论上允许 256TB 物理内存(Linux 在 x86-64 下通过 <a href="https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt">with 4 level page tables</a> 允许每个处理器 128TB 地址空间,Windows 192TB)。</p>
<p>由于物理内存的大小是受限制的,所以进程运行在自身的内存沙盒内 -- “虚拟内存地址(virtual address space)”,称作 <strong>虚拟内存(Virtual Memory)</strong>。</p>
<p><strong>字节的地址在这个虚拟地址空间内不再和处理器放在地址总线上的地址相同</strong>。因此必须建立转换数据结构和系统将虚拟地址空间中的字节映射到物理字节。</p>
<p>虚拟地址表示参见下图(<code>/proc/$PID/maps</code>):</p>
<p><img alt="Virtual Address Space Representation" src="static/upload/go-memory-allocator-visual-guide/5c6fbf14b9048569f7000004.png"></p>
<p>综上所述当 CPU 执行一个指令需要引用内存地址时。首先将在 VMA(Virtual Memory Areas)中的逻辑地址转换为线性地址。这个转换通过 <a href="https://zh.wikipedia.org/wiki/%E5%86%85%E5%AD%98%E7%AE%A1%E7%90%86%E5%8D%95%E5%85%83">MMU</a> 完成。</p>
<p><img alt="This is not a physical diagram, only a depiction. address translation process not included for simplicity" src="static/upload/go-memory-allocator-visual-guide/5c6fc2c1b9048569f7000005.png"></p>
<p>由于逻辑地址太大几乎很难独立的管理,所以引入术语 <strong>页(pages)</strong> 进行管理。当必要的分页操作被激活后,<strong>虚拟地址空间被分成更小的称作页的区域</strong>(大部分操作系统下是 4KB,可以修改)。页是虚拟内存中数据内存管理的最小单元。虚拟内存不存储任何内容,只是简单的将程序地址空间映射到底层物理内存之上。</p>
<p>独立的进程只能使用 VMA 作为他们的地址。<strong>所以当我们的程序需要更多 “堆内存(heap memory)时发生了什么?</strong></p>
<p>下图是简单的汇编代码用于分配更多的堆内存</p>
<p><img alt="A simple assembly code asking for more heap memory." src="static/upload/go-memory-allocator-visual-guide/5c6fc4f9b9048569f7000006.png"></p>
<p>下图描述堆内存的增长</p>
<p><img alt="heap memory increment" src="static/upload/go-memory-allocator-visual-guide/5c6fc53bb9048569f7000007.png"></p>
<p>应用程序通过系统调用 <a href="http://man7.org/linux/man-pages/man2/brk.2.html">brk</a>(<code>sbrk</code>/<code>mmap</code> 等)获得内存。内核仅更新堆 VMA 并调用它。</p>
<blockquote>
<p>当前时间点实际上不分配页帧且新页在物理内存中并不存在。这也是 VSZ 和 RSS 大小的不同点。</p>
</blockquote>
<h2 id="_2">内存分配器</h2>
<p>通过对“虚拟地址空间”基本了解和它对在堆分配的意义,内存分配器现在变得更加容易解释。</p>
<blockquote>
<p>如果堆上有足够的空间的满足我们代码的内存申请,内存分配器可以完成内存申请无需内核参与,否则将通过操作系统调用(<code>brk</code>)进行扩展堆,通常是申请一大块内存。(对于 <code>malloc</code> 大默认指的是大于 <code>MMAP_THRESHOLD</code> 个字节 - 128KB)。</p>
</blockquote>
<p>但是,内存分配器除了更新 <code>brk address</code> 还有其他职责。其中主要的一项就是如何<strong>减少</strong> <code>内部(internal)</code>和<code>外部(external)</code>碎片和如何快速分配当前块。考虑我们的程序以串行的方式(p1 到 p4)通过 <code>malloc(size)</code> 函数申请一块连续的内存然后通过 <code>free(pointer)</code> 函数进行释放。</p>
<p><img alt="An external fragmentation demonstration" src="static/upload/go-memory-allocator-visual-guide/5c6fcc0db9048569f7000008.png"></p>
<p>在 p4 阶段由于内存碎片化即使我们有足够的内存块依然无法满足申请的 6 个连续的内存块。</p>
<p><strong>所以我们该如何减少内存碎片化呢</strong> ?答案取决是使用哪种内存分配算法,也就是使用哪个底层库。</p>
<p>我们将简单看一下一个和 Go 内存分配器建模相近的内存分配器: <code>TCMalloc</code>。</p>
<h2 id="tcmalloc">TCMalloc</h2>
<p><a href="http://goog-perftools.sourceforge.net/doc/tcmalloc.html">TCMalloc</a> 的核心思想是将内存分为多个级别缩小锁的粒度。在 TCMalloc 内存管理内部分为两个部分:<strong>线程内存(thread memory)</strong>和<strong>页堆(page heap)</strong>。</p>
<h3 id="_3">线程内存</h3>
<p>每一个内存页都被分为多个固定分配大小规格的空闲列表(<code>free list</code>) 用于减少碎片化。这样每一个线程都可以获得一个用于无锁分配小对象的缓存,这样可以让并行程序分配小对象(<=32KB)非常高效。</p>
<p><img alt="Thread Cache (Each Thread gets this Thread Local Thread Cache)" src="static/upload/go-memory-allocator-visual-guide/5c6fd07bb9048569f7000009.png"></p>
<h3 id="_4">页堆</h3>
<p>TCMalloc 管理的堆由一组页组成,<strong>一组连续的页面被表示为 span</strong>。当分配的对象大于 32KB,将使用页堆(Page Heap)进行内存分配。</p>
<p><img alt="Page Heap (for span management)" src="static/upload/go-memory-allocator-visual-guide/5c6fd167b9048569f700000a.png"></p>
<p>当没有足够的空间分配小对象则会到页堆获取内存。如果页堆页没有足够的内存,则页堆会向操作系统申请更多的内存。</p>
<blockquote>
<p>Note: 即使 Go 的内存分配器最初是基于 TCMalloc,但是现在已经有很大的不同。</p>
</blockquote>
<h2 id="go">Go 内存分配器</h2>
<p>我们知道 Go 运行时(Go Runtime)调度器在调度时会将 <strong>Goroutines(G)</strong> 绑定到 <strong>逻辑处理器(P)(Logical Processors)</strong> 运行。类似的,Go 实现的 TCMalloc 将内存页(Memory Pages)分为 67 种不同大小规格的块。 </p>
<blockquote>
<p>如果你不熟悉 Go 的调度器可以先参见《<a href="https://povilasv.me/go-scheduler/">
Go scheduler: Ms, Ps & Gs
</a>》,然后继续阅读。</p>
</blockquote>
<p><img alt="Size Classes in Go" src="static/upload/go-memory-allocator-visual-guide/5c6fd34eb9048569f700000b.png"></p>
<p>如果页的规格大小为 1KB 那么 Go 管理粒度为 <strong>8192B</strong> 内存将被切分为 8 个像下图这样的块。</p>
<p><img alt="8 KB page divided into a size class of 1KB (In Go pages are maintained at the granularity of 8KB)" src="static/upload/go-memory-allocator-visual-guide/5c6fd4f7b9048569f700000c.png"></p>
<p>Go 中这些页通过 <strong>mspan</strong> 结构体进行管理。</p>
<h3 id="mspan">mspan</h3>
<p>简单的说,<code>mspan</code> 是一个包含页起始地址、页的 span 规格和页的数量的双端链表。</p>
<p><img alt="Illustrative Representation of a mspan in Go memory allocator" src="static/upload/go-memory-allocator-visual-guide/5c6fd878b9048569f700000d.png"></p>
<h3 id="mcache">mcache</h3>
<p>Go 像 TCMalloc 一样为每一个 <strong>逻辑处理器(P)(Logical Processors)</strong> 提供一个本地线程缓存(Local Thread Cache)称作 <strong>mcache</strong>,所以如果 Goroutine 需要内存可以直接从 <strong>mcache</strong> 中获取,由于在同一时间只有一个 Goroutine 运行在 <strong>逻辑处理器(P)(Logical Processors)</strong> 上,所以中间不需要任何锁的参与。</p>
<p><strong>mcache</strong> 包含所有大小规格的 <strong>mspan</strong> 作为缓存。</p>
<p><img alt="Illustrative Representation of a Relationship between P, mcache, and mspan in Go." src="static/upload/go-memory-allocator-visual-guide/5c6fda2eb9048569f700000e.png"></p>
<blockquote>
<p>由于每个 P 都拥有各自的 mcache,所以从 mcache 分配内存无需持有锁。</p>
</blockquote>
<p>对于每一种大小规格都有两个类型:</p>
<ol>
<li><strong>scan</strong> -- 包含指针的对象。</li>
<li><strong>noscan</strong> -- 不包含指针的对象。</li>
</ol>
<p>采用这种方法的好处之一就是进行垃圾回收时 <strong>noscan</strong> 对象无需进一步扫描是否引用其他活跃的对象。</p>
<h4 id="mcache_1">mcache 的作用是什么?</h4>
<blockquote>
<p><code><=32K</code> 字节的对象直接使用相应大小规格的 <strong>mspan</strong> 通过 <code>mcache</code> 分配</p>
</blockquote>
<h4 id="mcache_2">当 mcache 没有可用空间时会发生什么?</h4>
<p>从 <strong>mcentral</strong> 的 mspans 列表获取一个新的所需大小规格的 <code>mspan</code>。</p>
<h3 id="mcentral">mcentral</h3>
<p><code>mcentral</code> 对象收集所有给定规格大小的 span。每一个 <code>mcentral</code> 都包含两个 mspan 的列表:</p>
<ol>
<li><strong>empty</strong> mspanList -- 没有空闲对象或 span 已经被 mcache 缓存的 span 列表</li>
<li><strong>nonempty</strong> mspanList -- 有空闲对象的 span 列表</li>
</ol>
<p><img alt="Illustrative Representation of a mcentral" src="static/upload/go-memory-allocator-visual-guide/5c6fdd13b9048569f700000f.png"></p>
<p>每一个 mcentral 结构体都维护在 <strong>mheap</strong> 结构体内。</p>
<h3 id="mheap">mheap</h3>
<blockquote>
<p>Go 使用 mheap 对象管理堆,只有一个全局变量。持有虚拟地址空间。</p>
</blockquote>
<p><img alt="Illustrative Representation of a mheap." src="static/upload/go-memory-allocator-visual-guide/5c6fdd76b9048569f7000010.png"></p>
<p>就上我们从上图看到的:<strong>mheap 存储了 mcentral 的数组</strong>。<strong>这个数组包含了各个的 span 的 mcentral</strong>。</p>
<div class="highlight"><pre><span></span><code><span class="nx">central</span><span class="w"> </span><span class="p">[</span><span class="nx">numSpanClasses</span><span class="p">]</span><span class="kd">struct</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nx">mcentral</span><span class="w"> </span><span class="nx">mcentral</span>
<span class="w"> </span><span class="nx">pad</span><span class="w"> </span><span class="p">[</span><span class="nx">sys</span><span class="p">.</span><span class="nx">CacheLineSize</span><span class="w"> </span><span class="nx">unsafe</span><span class="p">.</span><span class="nx">Sizeof</span><span class="p">(</span><span class="nx">mcentral</span><span class="p">{})</span><span class="o">%</span><span class="nx">sys</span><span class="p">.</span><span class="nx">CacheLineSize</span><span class="p">]</span><span class="kt">byte</span>
<span class="p">}</span>
</code></pre></div>
<blockquote>
<p>由于我们有各个规格的 span 的 mcentral,当一个 <strong>mcache</strong> 从 mcentral 申请 <strong>mspan</strong> 时,只需要在独立的 <strong>mcentral</strong> 级别中使用锁,所以其它任何 <strong>mcache</strong> 在同一时间申请不同大小规格的 <strong>mspan</strong> 将互不受影响可以正常申请。</p>
</blockquote>
<p>对齐填充(Padding)用于确保 mcentrals 以 <em>CacheLineSize</em> 个字节数分隔,所以每一个 <em>MCentral.lock</em> 都可以获取自己的缓存行(cache line),以避免<a href="https://en.wikipedia.org/wiki/False_sharing">伪共享(false sharing)</a>问题。</p>
<p>当 <strong>mcentral</strong> 列表空的时候会发生什么?<strong>mcentral</strong> 从 <strong>mheap</strong> 获取一系列页用于需要的大小规格的 span。</p>
<ul>
<li><strong>free[_MaxMHeapList]mSpanList</strong>:一个 <code>spanList</code> 数组。每一个 <code>spanList</code> 中的 <strong>mspan</strong> 包含 1 ~ 127(_MaxMHeapList - 1)个页。例如,<code>free[3]</code> 是一个包含 3 个页的 <strong>mspan</strong> 链表。<code>free</code> 表示 <code>free list</code>,表示未分配。对应 <code>busy list</code>。</li>
<li><strong>freelarge mSpanList</strong>:一个 <strong>mspan</strong> 的列表。每一个元素(mspan)的页数大于 127。通过 <code>mtreap</code> 结构体管理。对应 <code>busylarge</code>。</li>
</ul>
<blockquote>
<p>大于 32K 的对象被定义为大对象,直接通过 mheap 分配。这些大对象的申请是以一个全局锁为代价的,因此任何给定的时间点只能同时供一个 P 申请。</p>
</blockquote>
<h2 id="_5">对象分配流程</h2>
<ul>
<li>大于 32K 的大对象直接从 <strong>mheap</strong> 分配。</li>
<li>小于 16B 的使用 <strong>mcache</strong> 的微型分配器分配</li>
<li>对象大小在 16B ~ 32K 之间的的,首先通过计算使用的大小规格,然后使用 <strong>mcache</strong> 中对应大小规格的块分配</li>
<li>如果对应的大小规格在 <strong>mcache</strong> 中没有可用的块,则向 <strong>mcentral</strong> 申请</li>
<li>如果 <strong>mcentral</strong> 中没有可用的块,则向 <strong>mheap</strong> 申请,并<strong>根据 BestFit 算法找到最合适的 mspan</strong>。如果申请到的 <strong>mspan</strong> 超出申请大小,将会根据需求进行切分,以返回用户所需的页数。剩余的页构成一个新的 <strong>mspan</strong> 放回 <strong>mheap</strong> 的空闲列表。</li>
<li>
<p>如果 <strong>mheap</strong> 中没有可用 span,则向操作系统申请一系列新的页(最小 1MB)。</p>
<blockquote>
<p>但是 Go 会在操作系统分配超大的页(称作 arena)。分配一大批页会减少和操作系统通信的成本。</p>
</blockquote>
</li>
</ul>
<p><strong>所有在堆上的内存申请都来自 arena</strong>。让我们看看 arena 是什么。</p>
<h2 id="go_1">Go 虚拟内存</h2>
<p>让我们看一个简单的 Go 程序的内存情况</p>
<div class="highlight"><pre><span></span><code><span class="kd">func</span><span class="w"> </span><span class="nx">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">{}</span>
<span class="p">}</span>
</code></pre></div>
<p><img alt="process stats for a program" src="static/upload/go-memory-allocator-visual-guide/5c6fefc5b9048569f7000011.png"></p>
<p>从上面可以即使是一个简单的程序虚拟空间占用页大概 <code>~100MB</code> 左右,但是 RSS 仅仅占用 <code>696KB</code>。让我们先搞清楚这之间的差异。</p>
<p><img alt="map and smap stats." src="static/upload/go-memory-allocator-visual-guide/5c6ff023b9048569f7000012.png"></p>
<p>这里有一块内存区域大小在 ~ <code>2MB</code>、<code>64MB</code> 和 <code>32MB</code>。这些是什么?</p>
<h3 id="arena">Arena</h3>
<p>事实证明 Go 的虚拟内存布局中包含一系列 <strong>arenas</strong>。初始的堆映射是一个 <strong>arena</strong>,如 <code>64MB</code>(基于 go 1.11.5)。</p>
<p><img alt="current incremental arena size on a different system." src="static/upload/go-memory-allocator-visual-guide/5c6ff112b9048569f7000013.png"></p>
<p>所以当前内存根据我们的程序需要以小增量映射,并且初始于一个 arena(~64MB)。</p>
<p>请首先记住这些数字。主题开始改变。早期 Go 需要预先保留一个连续的虚拟地址,在一个 64-bit 的系统 arena 的大小是 512GB。(如果分配的足够大且 <strong>被 mmap 拒绝</strong> 会发生什么?)</p>
<p><strong>这些 arenas 就是我们所说的堆</strong>。在 Go 中每一个 arena 都以 <code>8192B</code> 的粒度的页进行管理。</p>
<p>下图表示一个 64MB 的 arena</p>
<p><img alt="Single arena ( 64 MB )." src="static/upload/go-memory-allocator-visual-guide/5c701deab9048569f7000014.png"></p>
<p>Go 同时存在其他两个块:<strong>span</strong> 和 <strong>bitmap</strong>。<strong>两者都在堆外分配并且包含每个 arena 的元数据</strong>。大多用于垃圾回收期间(所以我们就讨论到这)。</p>
<hr>
<p>在我们刚刚讨论的 Go 的内存分配策略种类里,只涉及到内存分配奇妙且多样性的冰山一角。</p>
<p>然而,Go 内存管理的一般思想是使用不同的内存结构为不同大小的对象使用不同的内存缓存级别来分配内存。将一个从操作系统接收的连续地址的块切分到多级缓存来减少锁的使用,同时根据指定的大小分配内存减少内存碎片以提高内存分配的效率和在内存释放之后加快 GC 运行的速度。</p>
<p>现在我们将通过下图结束 Go 内存分配可视化指南。</p>
<p><img alt="Visual Overview of Runtime Memory Allocator." src="static/upload/go-memory-allocator-visual-guide/5c70206cb9048569f7000015.png"></p>
<hr>
<p>翻译参考:</p>
<ul>
<li><a href="http://share.onlinesjtu.com/mod/tab/view.php?id=253">总线内部结构</a></li>
<li><a href="https://zh.wikipedia.org/zh/%E5%8A%A8%E6%80%81%E9%9A%8F%E6%9C%BA%E5%AD%98%E5%8F%96%E5%AD%98%E5%82%A8%E5%99%A8">DRAM</a></li>
<li><a href="https://zh.wikipedia.org/wiki/%E7%89%A9%E7%90%86%E5%9C%B0%E5%9D%80%E6%89%A9%E5%B1%95">PAE</a></li>
<li><a href="https://www.wikiwand.com/en/Byte_addressing">Byte addressing</a></li>
<li><a href="https://www.csie.ntu.edu.tw/~wcchen/asm98/asm/proj/b85506061/chap2/overview.html">内存管理</a></li>
</ul>
<hr>
<ul>
<li>原文链接 <a href="https://blog.learngoprogramming.com/a-visual-guide-to-golang-memory-allocator-from-ground-up-e132258453ed">A visual guide to Go Memory Allocator from scratch (Golang)</a></li>
<li>GitHub 地址:<a href="https://github.com/coldnight/go-memory-allocator-visual-guide">go-memory-allocator-visual-guide</a></li>
</ul>《敏捷革命》读书笔记2019-02-15T00:00:00+08:002019-02-15T00:00:00+08:00coldtag:www.linuxzen.com,2019-02-15:/scrum-read-notes.html<div class="section" id="section-2">
<h2>基本过程</h2>
<div class="section" id="section-3">
<h3>准备</h3>
<ol class="arabic simple">
<li>挑选一位 <strong>产品负责人(Product Owner)</strong> :负责沟通客户并根据客户反馈规划待办事项</li>
<li>挑选 <strong>团队</strong> :小而精,3 ~ 9 人</li>
<li>挑选 <strong>Scrum 主 …</strong></li></ol></div></div><div class="section" id="section-2">
<h2>基本过程</h2>
<div class="section" id="section-3">
<h3>准备</h3>
<ol class="arabic simple">
<li>挑选一位 <strong>产品负责人(Product Owner)</strong> :负责沟通客户并根据客户反馈规划待办事项</li>
<li>挑选 <strong>团队</strong> :小而精,3 ~ 9 人</li>
<li>挑选 <strong>Scrum 主管(Scrum Master)</strong> :培训 Scrum 确保 Scrum 正确运用,消除团队障碍</li>
<li>确定 <strong>冲刺(Sprint)</strong> 周期,不要超过一个月,最好 1 ~ 2 周</li>
</ol>
</div>
<div class="section" id="section-4">
<h3>执行</h3>
<ol class="arabic">
<li><p class="first">产品负责人拟定 <strong>待办事项</strong> ,并确定优先级</p>
</li>
<li><p class="first">让实际开发的团队评估和改进待办事项</p>
</li>
<li><p class="first">通过 <strong>冲刺规划会</strong> 规划接下来一个冲刺要完成的待办事项,并将借助 <strong>看板</strong> 工具将工作透明化</p>
<p>看板可分为如下几个项:</p>
<ol class="arabic simple">
<li>To do -- 当前冲刺需要完成的任务</li>
<li>In progress -- 当前冲刺正在进行的任务</li>
<li>Done -- 当前冲刺已经完成的任务</li>
</ol>
</li>
<li><p class="first">冲刺开始后需要进行 <strong>每日站会</strong> ,时间不应高过 15 分钟,Scrum 主管向团队成员提出下列问题</p>
<ol class="arabic simple">
<li>距离上次站会之后到现在你做了什么去帮助团队完成冲刺</li>
<li>距离现在到下次站会之前你打算做什么去帮助团队完成冲刺</li>
<li>什么因素阻碍了团队完成冲刺</li>
</ol>
<p>站会应在每天的固定时间点,团队所有成员必须准点参加。</p>
</li>
<li><p class="first">每次冲刺结束之前展示 <strong>冲刺成果</strong> ,面向展示的对象主要包括:利益相关者、管理人员与客户</p>
</li>
</ol>
</div>
<div class="section" id="section-5">
<h3>回顾</h3>
<p>每次冲刺结束之后下一个冲刺开始之前要举行 <strong>冲刺回顾会议</strong> ,会议主要讨论以下内容:</p>
<ul class="simple">
<li>展示之前冲刺中创造的成果</li>
<li>总结冲刺中哪些做的好,哪些需要做的更好,找出真正的障碍并拿出勇气将障碍摆在台面上</li>
<li><strong>确定一个最值得改善的地方,将其设定为下一个冲刺的首要任务(关键)</strong></li>
<li>计算快乐指标(见下文量化快乐)</li>
</ul>
</div>
<div class="section" id="section-6">
<h3>下一个冲刺</h3>
<p>上一个冲刺结束后,立即开始新的冲刺。</p>
</div>
<div class="section" id="section-7">
<h3>辅助工具</h3>
<ul class="simple">
<li><a class="reference external" href="https://zh.wikipedia.org/wiki/%E7%9C%8B%E6%9D%BF_(%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91)">看板</a></li>
<li><a class="reference external" href="https://zh.wikipedia.org/wiki/%E7%87%83%E5%B0%BD%E5%9B%BE">燃尽图</a></li>
</ul>
</div>
</div>
<div class="section" id="section-8">
<h2>传达的理念</h2>
<div class="section" id="section-9">
<h3>节奏</h3>
<p>Scrum 流程的核心是节奏,每天准点的站会,固定周期的冲刺,冲刺之后的回顾都将我们带入一种节奏中。</p>
</div>
<div class="section" id="section-10">
<h3>守破离</h3>
<p>要先学习规则和形式,掌握之后再进行创新,最终摆脱规则和形式。</p>
</div>
<div class="section" id="section-11">
<h3>强调团队而非个人</h3>
<ul class="simple">
<li>拒绝产生英雄式人物,如果一个团队的领导者在休假时还要确保办公室一切正常,那么就说明他没有管理好自己的团队</li>
<li>能够自主决策,最大化地提升人的自由、能力和创造力。</li>
<li>不为失败指责他人,客观的找出问题整个团队一起完善</li>
<li>强调团队绩效而非个人绩效</li>
</ul>
</div>
<div class="section" id="section-12">
<h3>拒绝浪费</h3>
<ul class="simple">
<li>鉴于人类单线程生物,不要同时执行多项任务,会浪费精力导致最终所有事情都做不好 -- 同时执行多项任务会让你变的愚蠢</li>
<li>如果事情做到一半就放弃了等同与什么也没做 -- 半途而废等于没做</li>
<li>应该一边做一边测试,并及时改正测试出来的错误,而不是等整个产品成型后再交付测试导致返工造成浪费 -- 一次性把事情做好</li>
<li>延长工时不会提高效率,反而会陷入恶性循环,造成浪费</li>
</ul>
</div>
<div class="section" id="section-13">
<h3>评估使用点数替换时间</h3>
<p>传统的项目排期会评估某个任务的截止时间或者所需时长,但是在敏捷中应该评估任务的点数,并有以下几个要点:</p>
<ul>
<li><p class="first">点数不使用常规的顺序的数字,而是使用 <a class="reference external" href="https://zh.wikipedia.org/wiki/%E6%96%90%E6%B3%A2%E9%82%A3%E5%A5%91%E6%95%B0%E5%88%97">斐波那契数列(黄金分割数列)</a> ,好处就是可以明确的分辨不同点数之间的差异,考虑如下两列数列的差异</p>
<table border="1" class="docutils">
<colgroup>
<col width="45%" />
<col width="55%" />
</colgroup>
<tbody valign="top">
<tr><td>正常顺序</td>
<td>斐波那契</td>
</tr>
<tr><td>1</td>
<td>2</td>
</tr>
<tr><td>2</td>
<td>3</td>
</tr>
<tr><td>3</td>
<td>5</td>
</tr>
<tr><td>4</td>
<td>8</td>
</tr>
<tr><td>5</td>
<td>13</td>
</tr>
</tbody>
</table>
<p>斐波那契数列能更好分辨不同级别的点数之间的差异,如 13 和 8 之间的差异,就比 5 和 4 之间的差异要好分辨的多。</p>
</li>
<li><p class="first">采用类似 <a class="reference external" href="https://zh.wikipedia.org/wiki/%E5%BE%B7%E5%B0%94%E8%8F%B2%E6%B3%95">德尔菲法</a> 的方式评估任务点数,避免相互影响</p>
<p>方法是类似打扑克牌的方式进行评估</p>
<ol class="arabic simple">
<li>参与评估每个人都拥有 2 3 5 8 13 这 5 张牌</li>
<li>对每一个需要评估的任务所有人都给出自己的点数但不公开</li>
<li>所有人都评估完成后给出点数</li>
<li>得出结果,有以下几种情况<ol class="arabic">
<li>所有人点数相近,取平均值</li>
<li>差距较大的情况下,找出差距最大的两人分别阐述原因,然后重新出牌,直到点数相近,取平均值</li>
</ol>
</li>
</ol>
<p>因为人类本身就有从众的缺点,并且很容易受 <a class="reference external" href="https://zh.wikipedia.org/wiki/%E6%99%95%E8%BD%AE%E6%95%88%E5%BA%94">光环效应</a> 的影响导致判断偏差,采用这种方法评估能很好的规避这些缺点。</p>
</li>
</ul>
<p>以此评估就可以统计出每个冲刺完成的任务点数,实践 Scrum 后团队完成的点数会在初期成倍增加。</p>
<div class="section" id="section-14">
<h4>免费更换需求</h4>
<p>需求可以免费更换,这里免费指对客户免费,在自有产品中可以换成自由更换,但自由更换并不代表没有代价,要从已经安排进去的待办事项中替换相同点数的事项。</p>
</div>
</div>
<div class="section" id="section-15">
<h3>快乐是第一生产力</h3>
<p>这里的快乐是指可以让团队提高效率的快乐,而非享乐的快乐,比如</p>
<ul class="simple">
<li>提高团队运作的透明度,不应该有秘密小集团、秘密日程和其他什么秘而不宣的事情</li>
<li>给予团队中的每个人自主权,让每个人都受到尊重</li>
<li>承认只有糟糕的制度没有糟糕的团队</li>
<li>团队成员之间相互信任,踢出让其他人不快乐的成员等等</li>
<li>扁平化</li>
<li>增强团队成员之间的联系(羁绊的力量)</li>
</ul>
<p>这里要避免经过一段时间的进步之后安于现状,要有“聪明的傻瓜”勇于刺破“快乐的泡沫”。</p>
<div class="section" id="section-16">
<h4>量化快乐</h4>
<p>在每次冲刺结束后应统计成员的快乐指标,参与冲刺的每位成员都需要回答以下 4 个问题</p>
<ul class="simple">
<li>你对自己在公司的角色感觉如何?请以 1 ~ 5 分加以评价</li>
<li>你对公司整体情况感觉如何?请以 1 ~ 5 分加以评价</li>
<li>为什么会有这种感受?</li>
<li>在下一个冲刺阶段中,什么事情会让你感到更快乐?</li>
</ul>
<p>通过计算评价分可以得出整个团队平均快乐指标。</p>
</div>
<div class="section" id="mvp">
<h4>MVP 最简化可行产品</h4>
<ul class="simple">
<li>一个产品 80% 的价值来自 20% 的功能</li>
<li>产品负责人列出待办事项并找出其中对用户最有价值的</li>
<li>及时犯错:有些错误早犯可以尽量减少给别人造成的伤害,而且以后也可以想办法避免</li>
</ul>
</div>
</div>
</div>
<div class="section" id="section-17">
<h2>总结</h2>
<p>本书引用大量的实践 Scrum 的团队的经历和研究结果,来阐述敏捷的力量。同时也通过大量的例子表明 Scrum 不仅仅可以用在软件开发,同时也适用于教育、政府、金融等其他行业。
总之说服力(洗脑效果)非常强。</p>
</div>
<div class="section" id="mvp-1">
<h2>快速入门敏捷和理解 MVP</h2>
<ul class="simple">
<li><a class="reference external" href="https://cloud.tencent.com/developer/article/1004881">你大概走了假敏捷:认真说说敏捷的实现和问题(手绘版)</a></li>
<li><a class="reference external" href="http://www.woshipm.com/ucd/774702.html">MVP与精益创业</a></li>
</ul>
</div>
通过 acme.sh 获取 Let's Encrypt 免费证书2019-02-12T00:00:00+08:002019-02-12T00:00:00+08:00coldtag:www.linuxzen.com,2019-02-12:/use-acme-sh-to-issue-letsencrypt-cert.html<div class="section" id="nginx-webroot">
<h2>配置 Nginx 正确处理 Webroot 验证</h2>
<p>在证书签发过程中 Let's Encrypt 会验证你拥有当前域名,最基本的方式在你的网站根目录创建一个文件 …</p></div><div class="section" id="nginx-webroot">
<h2>配置 Nginx 正确处理 Webroot 验证</h2>
<p>在证书签发过程中 Let's Encrypt 会验证你拥有当前域名,最基本的方式在你的网站根目录创建一个文件,并通过域名在外部进行请求,如能请求到则认为你拥有该网站的控制权。假设你有一个域名 <tt class="docutils literal">example.com</tt>, 验证步骤大体如下:</p>
<ol class="arabic simple">
<li>通过工具在网站根目录下创建 <tt class="docutils literal"><span class="pre">.well-known/acme-challenge/some-random-letters</span></tt></li>
<li>工具将创建的路径告知 Let's Encrypt</li>
<li>Let's Encrypt 通过域名请求该文件,如 <tt class="docutils literal"><span class="pre">http://example.com/.well-known/acme-challenge/some-random-letters</span></tt></li>
<li>若能请求到则确认拥有该网站的控制权颁发证书,否则拒绝颁发</li>
</ol>
<p>为了简化多个域名颁发证书需指定不同的 Webroot,我们可以将所有域名的验证统一放在一个目录下,并新增一个配置片段供需要启用 HTTPS 的网站引用,新增 <tt class="docutils literal"><span class="pre">/etc/nginx/snippets/letsencrypt-acme-challenge.conf</span></tt> ,并填充如下内容</p>
<div class="highlight"><pre><span></span><span class="c1">#############################################################################</span>
<span class="c1"># Configuration file for Let's Encrypt ACME Challenge location</span>
<span class="c1">#############################################################################</span>
<span class="c1">#</span>
<span class="c1"># This config enables to access /.well-known/acme-challenge/xxxxxxxxxxx</span>
<span class="c1"># on all our sites (HTTP), including all subdomains.</span>
<span class="c1"># This is required by ACME Challenge (webroot authentication).</span>
<span class="c1"># You can check that this location is working by placing ping.txt here:</span>
<span class="c1"># /var/www/letsencrypt/.well-known/acme-challenge/ping.txt</span>
<span class="c1"># And pointing your browser to:</span>
<span class="c1"># http://xxx.domain.tld/.well-known/acme-challenge/ping.txt</span>
<span class="c1">#</span>
<span class="c1"># Sources:</span>
<span class="c1"># https://community.letsencrypt.org/t/howto-easy-cert-generation-and-renewal-with-nginx/3491</span>
<span class="c1">#</span>
<span class="c1">#############################################################################</span>
<span class="c1"># Rule for legitimate ACME Challenge requests (like /.well-known/acme-challenge/xxxxxxxxx)</span>
<span class="c1"># We use ^~ here, so that we don't check other regexes (for speed-up). We actually MUST cancel</span>
<span class="c1"># other regex checks, because in our other config files have regex rule that denies access to files with dotted names.</span>
<span class="k">location</span><span class="w"> </span><span class="s">^~</span><span class="w"> </span><span class="s">/.well-known/acme-challenge/</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1"># Set correct content type. According to this:</span>
<span class="w"> </span><span class="c1"># https://community.letsencrypt.org/t/using-the-webroot-domain-verification-method/1445/29</span>
<span class="w"> </span><span class="c1"># Current specification requires "text/plain" or no content header at all.</span>
<span class="w"> </span><span class="c1"># It seems that "text/plain" is a safe option.</span>
<span class="w"> </span><span class="kn">default_type</span><span class="w"> </span><span class="s">"text/plain"</span><span class="p">;</span>
<span class="w"> </span><span class="c1"># This directory must be the same as in /etc/letsencrypt/cli.ini</span>
<span class="w"> </span><span class="c1"># as "webroot-path" parameter. Also don't forget to set "authenticator" parameter</span>
<span class="w"> </span><span class="c1"># there to "webroot".</span>
<span class="w"> </span><span class="c1"># Do NOT use alias, use root! Target directory is located here:</span>
<span class="w"> </span><span class="c1"># /var/www/common/letsencrypt/.well-known/acme-challenge/</span>
<span class="w"> </span><span class="kn">root</span><span class="w"> </span><span class="s">/var/www/letsencrypt</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1"># Hide /acme-challenge subdirectory and return 404 on all requests.</span>
<span class="c1"># It is somewhat more secure than letting Nginx return 403.</span>
<span class="c1"># Ending slash is important!</span>
<span class="k">location</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">/.well-known/acme-challenge/</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">return</span><span class="w"> </span><span class="mi">404</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>在需要启用验证的网站的 Nginx 配置中增加如下内容即可</p>
<div class="highlight"><pre><span></span><span class="k">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">...</span>
<span class="w"> </span><span class="s">include</span><span class="w"> </span><span class="s">/etc/nginx/snippets/letsencrypt-acme-challenge.conf</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>创建相应目录重启 Nginx</p>
<div class="highlight"><pre><span></span><span class="k">sudo</span><span class="w"> </span><span class="s">mkdir</span><span class="w"> </span><span class="s">-p</span><span class="w"> </span><span class="s">/var/www/letsencrypt/</span>
<span class="s">sudo</span><span class="w"> </span><span class="s">chown</span><span class="w"> </span>$<span class="s">(whoami).</span>$<span class="s">(whoami)</span><span class="w"> </span><span class="s">/var/www/letsencrypt/</span>
<span class="s">sudo</span><span class="w"> </span><span class="s">/etc/init.d/nginx</span><span class="w"> </span><span class="s">configtest</span><span class="w"> </span><span class="s">&&</span><span class="w"> </span><span class="s">sudo</span><span class="w"> </span><span class="s">/etc/init.d/nginx</span><span class="w"> </span><span class="s">reload</span>
</pre></div>
</div>
<div class="section" id="section-1">
<h2>获取证书</h2>
<ol class="arabic">
<li><p class="first">安装 <a class="reference external" href="https://github.com/Neilpang/acme.sh">acme.sh</a></p>
<div class="highlight"><pre><span></span>curl<span class="w"> </span>https://get.acme.sh<span class="w"> </span><span class="p">|</span><span class="w"> </span>sh
</pre></div>
</li>
<li><p class="first">获取证书</p>
<div class="highlight"><pre><span></span>acme.sh<span class="w"> </span>--issue<span class="w"> </span>-d<span class="w"> </span>example.com<span class="w"> </span>-d<span class="w"> </span>www.example.com<span class="w"> </span>-w<span class="w"> </span>/var/www/letsencrypt
</pre></div>
</li>
</ol>
</div>
<div class="section" id="section-2">
<h2>安装证书</h2>
<p>获取的证书因为在用户家目录下,所以不能直接使用,需要通过如下命令安装到系统中</p>
<div class="highlight"><pre><span></span>sudo<span class="w"> </span>mkdir<span class="w"> </span>-p<span class="w"> </span>/etc/nginx/certs/example.com
sudo<span class="w"> </span>chown<span class="w"> </span>root.<span class="k">$(</span>whoami<span class="k">)</span><span class="w"> </span>/etc/nginx/certs/example.com
sudo<span class="w"> </span>chmod<span class="w"> </span>g+w<span class="w"> </span>/etc/nginx/certs/example.com
acme.sh<span class="w"> </span>--install-cert<span class="w"> </span>-d<span class="w"> </span>example.com<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--cert-file<span class="w"> </span>/etc/nginx/certs/example.com/cert.pem<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--key-file<span class="w"> </span>/etc/nginx/certs/example.com/key.pem<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--fullchain-file<span class="w"> </span>/etc/nginx/certs/example.com/fullchain.pem<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--reloadcmd<span class="w"> </span><span class="s2">"service nginx reload"</span>
</pre></div>
<p>安装后可以配置 Nginx 启用 HTTPS</p>
<div class="highlight"><pre><span></span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">80</span><span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span>www.example.com<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span>example.com<span class="p">;</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="m">301</span><span class="w"> </span>https://<span class="nv">$server_name$request_uri</span><span class="p">;</span>
<span class="o">}</span>
server<span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">443</span><span class="w"> </span>ssl<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span>www.example.com<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span>example.com<span class="p">;</span>
<span class="w"> </span>ssl_certificate<span class="w"> </span>/etc/nginx/certs/example.com/fullchain.pem<span class="p">;</span>
<span class="w"> </span>ssl_certificate_key<span class="w"> </span>/etc/nginx/certs/example.com/key.pem<span class="p">;</span>
<span class="w"> </span>ssl_protocols<span class="w"> </span>TLSv1<span class="w"> </span>TLSv1.1<span class="w"> </span>TLSv1.2<span class="p">;</span>
<span class="w"> </span>ssl_prefer_server_ciphers<span class="w"> </span>on<span class="p">;</span>
<span class="w"> </span>ssl_dhparam<span class="w"> </span>/etc/ssl/certs/dhparam.pem<span class="p">;</span>
<span class="w"> </span>ssl_ciphers<span class="w"> </span><span class="s1">'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'</span><span class="p">;</span>
<span class="w"> </span>ssl_session_timeout<span class="w"> </span>1d<span class="p">;</span>
<span class="w"> </span>ssl_session_cache<span class="w"> </span>shared:SSL:50m<span class="p">;</span>
<span class="w"> </span>ssl_stapling<span class="w"> </span>on<span class="p">;</span>
<span class="w"> </span>ssl_stapling_verify<span class="w"> </span>on<span class="p">;</span>
<span class="w"> </span>add_header<span class="w"> </span>Strict-Transport-Security<span class="w"> </span>max-age<span class="o">=</span><span class="m">15768000</span><span class="p">;</span>
<span class="w"> </span>access_log<span class="w"> </span>/var/log/nginx/www.example.com.log<span class="p">;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>/var/www/html<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.html<span class="w"> </span>index.htm<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>include<span class="w"> </span>/etc/nginx/snippets/letsencrypt-acme-challenge.conf<span class="p">;</span>
<span class="o">}</span>
</pre></div>
</div>
<div class="section" id="section-3">
<h2>参考</h2>
<ul class="simple">
<li><a class="reference external" href="https://community.letsencrypt.org/t/how-to-nginx-configuration-to-enable-acme-challenge-support-on-all-http-virtual-hosts/5622">https://community.letsencrypt.org/t/how-to-nginx-configuration-to-enable-acme-challenge-support-on-all-http-virtual-hosts/5622</a></li>
</ul>
</div>
Docker Snippets2019-02-11T17:30:00+08:002019-02-11T17:30:00+08:00coldtag:www.linuxzen.com,2019-02-11:/docker-snippets.html<p class="first last">分享关于 Docker 的一些坑</p>
<div class="section" id="docker-iptables">
<h2>Docker 和 iptables</h2>
<p>如果你用 Docker 启动了一个服务暴露了一个端口,同时你又想通过 iptables 限制该端口的访问,这时候你按照经验往 <tt class="docutils literal">INPUT</tt> 链插入一条规则发现并没有卵用,你应该看看 Docker 的官方文档 <a class="reference external" href="https://docs.docker.com/network/iptables/#add-iptables-policies-before-dockers-rules">Docker and iptables</a> 。这是因为 Docker 使用 iptables 提供网络隔离,只有在 <tt class="docutils literal">DOCKER</tt> 和 <tt class="docutils literal"><span class="pre">DOCKER-USER</span></tt> 链上的规则才对容器暴露的端口生效,官方推荐用户在 <tt class="docutils literal"><span class="pre">DOCKER-USER</span></tt> 链上新增自定义规则。</p>
</div>
<div class="section" id="docker">
<h2>使用 Docker 多阶段构建</h2>
<p>自 Docker 17.05 Docker 开始支持多阶段构建,使用 Docker 多阶段构建有两个好处:</p>
<ol class="arabic simple">
<li>优化镜像大小</li>
<li>在镜像中排除源代码</li>
</ol>
<p>比如一个典型的 Go 应用如果想要在镜像构建过程中编译,但是又不想镜像中包含源代码可以按照如下方式编写 <tt class="docutils literal">Dockerfile</tt></p>
<div class="highlight"><pre><span></span><span class="k">FROM</span><span class="w"> </span><span class="s">golang:1.10</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="s">intermediate</span>
<span class="k">ENV</span><span class="w"> </span>PROJECT_PATH<span class="w"> </span><span class="nv">$GOPATH</span>/src/github.com/coldnight/foobar
<span class="k">ADD</span><span class="w"> </span>.<span class="w"> </span><span class="nv">$PROJECT_PATH</span>/
<span class="k">WORKDIR</span><span class="w"> </span><span class="s">$PROJECT_PATH</span>
<span class="k">RUN</span><span class="w"> </span>go<span class="w"> </span>install
<span class="k">FROM</span><span class="w"> </span><span class="s">alpine:latest</span>
<span class="k">COPY</span><span class="w"> </span>--from<span class="o">=</span>intermediate<span class="w"> </span>/go/bin/foobar<span class="w"> </span>/go/bin/foobar
<span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="s2">"/go/bin/foobar"</span><span class="p">]</span>
</pre></div>
<p>具体使用方式参见官方文档 <a class="reference external" href="https://docs.docker.com/develop/develop-images/multistage-build/">Use multi-stage builds</a> 。</p>
</div>
<div class="section" id="cmd-vs-entrypoint">
<h2>CMD vs ENTRYPOINT</h2>
<p>我在一开始使用 Docker 时无法确定该使用 CMD 和 ENTRYPOINT 中的哪一个,其实很简单 CMD 可以在 <tt class="docutils literal">docker run</tt> 时被覆盖,而 ENTRYPOINT 不可以,比如如下 <tt class="docutils literal">Dockerfile</tt>:</p>
<div class="highlight"><pre><span></span><span class="k">FROM</span><span class="w"> </span><span class="s">alpine:latest</span>
<span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="err">'/bi</span><span class="kc">n</span><span class="err">/sh'</span><span class="p">]</span>
</pre></div>
<p>假设通过如下命令构建镜像:</p>
<div class="highlight"><pre><span></span>$<span class="w"> </span>docker<span class="w"> </span>build<span class="w"> </span>.<span class="w"> </span>-t<span class="w"> </span>demo
</pre></div>
<p>如果通过如下命令运行容器则默认启动 <tt class="docutils literal">/bin/sh</tt>:</p>
<div class="highlight"><pre><span></span>$<span class="w"> </span>docker<span class="w"> </span>run<span class="w"> </span>-it<span class="w"> </span>demo
</pre></div>
<p>但如果想覆盖则可以通过如下方式运行容器:</p>
<div class="highlight"><pre><span></span>$<span class="w"> </span>docker<span class="w"> </span>run<span class="w"> </span>-it<span class="w"> </span>demo<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'hello, world'</span>
</pre></div>
<p>如果将 Dockerfile 中 CMD 替换为 ENTRYPOINT 则无法通过上述方式覆盖默认的启动命令。</p>
</div>
<div class="section" id="docker-web">
<h2>Docker Web 管理系统</h2>
<p>觉得手动敲命令管理本地 Docker 麻烦?可以试试本地安装一个 <a class="reference external" href="https://github.com/portainer/portainer">portainer</a> ,通过 Web 界面管理本地 Docker。</p>
</div>
Go 1.5 内存清除汇编源码注释2019-02-11T00:00:00+08:002019-02-11T00:00:00+08:00coldtag:www.linuxzen.com,2019-02-11:/golang-105-memclr-assembly-comment.html<p class="first last">Go 为了实现内存复用会将已经回收到 cache 的内存清除,清除逻辑使用汇编实现,本文是基于自己的理解进行的注释。</p>
<p>最近在阅读 Go1.5 的源码,发现源码中多处调用了了 <tt class="docutils literal">memclr</tt> ,进一步深入了解发现原来 Go 为了实现内存复用会将已经回收到 <tt class="docutils literal">cache</tt> 的内存清除,清除逻辑使用汇编实现,本文是基于自己的理解进行的注释。</p>
<p><strong>注意</strong>:</p>
<ol class="arabic simple">
<li>最新版的 Go 中(1.11)该函数已经改名为 <tt class="docutils literal">memclrNoHeapPointers</tt></li>
<li>我自己添加的注释以 <tt class="docutils literal">//#</tt> 开始</li>
<li>Go 针对不同的平台实现了不同的内存清除,这里仅对 <tt class="docutils literal">amd64</tt> 平台下的实现进行注释:</li>
</ol>
<p>源文件: <a class="reference external" href="https://github.com/golang/go/blob/release-branch.go1.5/src/runtime/memclr_amd64.s">https://github.com/golang/go/blob/release-branch.go1.5/src/runtime/memclr_amd64.s</a></p>
<div class="highlight"><pre><span></span>// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
#include "textflag.h"
// NOTE: Windows externalthreadhandler expects memclr to preserve DX.
// void runtime·memclr(void* ptr, uintptr n)
//# - TEXT 在 Plan9 中用于声明函数
//# - runtime·memclr 为函数名,中间的·不是普通的点在 Mac 下通过 Option+Shift+9 打出,· 前是包名后面是函数名
//# - SB(伪寄存器)全局静态基指针,用来声明函数或全局变量(此处用来声明函数)
//# - NOSPLIT 定义在 https://github.com/golang/go/blob/master/src/runtime/textflag.h
//# - $0-16:
//# + $0 栈帧大小为0(局部变量+可能需要的额外调用函数的参数空间总大小,不包括调用其他函数时的 ret address 的大小)
//# + 16 参数基返回值的大小(16 表示两个双四字的参数)
TEXT runtime·memclr(SB), NOSPLIT, $0-16
//# FP(伪寄存器):通过 `symbol+offset(FP)` 的方式引用输入参数
//# symbol 没有任何用,只是增加可读性,但不能省略
//# FP 指向整个栈帧的底部的 BP 寄存器
MOVQ ptr+0(FP), DI //# 第一个参数移动到 DI 寄存器(DI 目标索引寄存器)
MOVQ n+8(FP), BX //# 第二个参数移动到 BX 寄存器(BX 为基址寄存器,用于内存寻址)
XORQ AX, AX //# 清零 AX 寄存器(AX 为累加寄存器)
// MOVOU seems always faster than REP STOSQ.
tail:
TESTQ BX, BX // set ZF to 1 if n is 0
JEQ _0 // jump to _0 if ZF == 1(returns)
CMPQ BX, $2
JBE _1or2 // jump to _1or2 if n <= 2
CMPQ BX, $4
JBE _3or4 // jump to _3or4 if n > 2 and n <= 4
CMPQ BX, $8
// ...
JBE _5through8
CMPQ BX, $16
JBE _9through16
//# 大于 16 开始使用 128 位寄存器 X0,
//# PXOR 将 X0 寄存器置为 0
PXOR X0, X0
CMPQ BX, $32
JBE _17through32
CMPQ BX, $64
JBE _33through64
CMPQ BX, $128
JBE _65through128
CMPQ BX, $256
JBE _129through256
// TODO: use branch table and BSR to make this just a single dispatch
// TODO: for really big clears, use MOVNTDQ.
//# 大于 256 则通过循环
loop:
//# MOVOU 相当于 AT&T/Intel 的 MOVDQU -- 移动非对齐的双四字
//# X0 相当与 AT&T/Intel 的 SSE 新增的 %xmm0(128位元暂存器)
//# 参见 https://zh.wikipedia.org/wiki/SSE
MOVOU X0, 0(DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, 64(DI)
MOVOU X0, 80(DI)
MOVOU X0, 96(DI)
MOVOU X0, 112(DI)
MOVOU X0, 128(DI)
MOVOU X0, 144(DI)
MOVOU X0, 160(DI)
MOVOU X0, 176(DI)
MOVOU X0, 192(DI)
MOVOU X0, 208(DI)
MOVOU X0, 224(DI)
MOVOU X0, 240(DI)
SUBQ $256, BX //# 递减 BX
ADDQ $256, DI //# 递增 DI
CMPQ BX, $256
JAE loop //# 如果 BX 依然大于 256 则继续循环
JMP tail //# 否则进入 tail
_1or2:
MOVB AX, (DI)
MOVB AX, -1(DI)(BX*1)
RET
_0:
RET
_3or4:
MOVW AX, (DI)
MOVW AX, -2(DI)(BX*1)
RET
_5through8:
MOVL AX, (DI)
MOVL AX, -4(DI)(BX*1)
RET
_9through16:
MOVQ AX, (DI)
MOVQ AX, -8(DI)(BX*1)
RET
_17through32:
MOVOU X0, (DI)
MOVOU X0, -16(DI)(BX*1)
RET
_33through64:
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
_65through128:
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, -64(DI)(BX*1)
MOVOU X0, -48(DI)(BX*1)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
_129through256:
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, 64(DI)
MOVOU X0, 80(DI)
MOVOU X0, 96(DI)
MOVOU X0, 112(DI)
MOVOU X0, -128(DI)(BX*1)
MOVOU X0, -112(DI)(BX*1)
MOVOU X0, -96(DI)(BX*1)
MOVOU X0, -80(DI)(BX*1)
MOVOU X0, -64(DI)(BX*1)
MOVOU X0, -48(DI)(BX*1)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
</pre></div>
<p>在此期间参阅了大量的资料,最大的坑就是 Go 使用的汇编是 Plan9而非常见的 x86 汇编,参考资料如下:</p>
<ul class="simple">
<li><a class="reference external" href="https://quasilyte.github.io/blog/post/go-asm-complementary-reference/">https://quasilyte.github.io/blog/post/go-asm-complementary-reference/</a></li>
<li><a class="reference external" href="https://gocn.vip/article/733">https://gocn.vip/article/733</a></li>
<li><a class="reference external" href="https://github.com/golang/arch/blob/master/x86/x86.csv">https://github.com/golang/arch/blob/master/x86/x86.csv</a></li>
<li><a class="reference external" href="https://golang.org/doc/asm">https://golang.org/doc/asm</a></li>
</ul>
修改本地 Git 历史2018-11-21T16:33:00+08:002018-11-21T16:33:00+08:00coldtag:www.linuxzen.com,2018-11-21:/xiu-gai-ben-di-git-li-shi.html<blockquote>
<p>很早之前一篇发表在内部的文章,抽时间整理了一下发布出来。</p>
</blockquote>
<p><strong>以下操作会修改提交历史, 可能会造成一些不可恢复的问 …</strong></p><blockquote>
<p>很早之前一篇发表在内部的文章,抽时间整理了一下发布出来。</p>
</blockquote>
<p><strong>以下操作会修改提交历史, 可能会造成一些不可恢复的问题, <em>不是</em> 下面情况不要这么操作</strong></p>
<ol>
<li>基于 GitHub Fork -> Pull Request 流程仅针对 Fork 后的仓库进行操作</li>
<li>非第一种情况的前提是当前修改的提交还未提交到远端</li>
</ol>
<p><strong>操作下面命令前最好先备份</strong></p>
<h2 id="_1">修改上一条提交的信息</h2>
<p>有时候我们在用 Git 提交后发现提交信息(commit message)不是我们预期的内容(错别字或描述错误等), 这个时候我们可能想修改一下上一条提交的信息, 有两种方式可以进行:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>git<span class="w"> </span>commit<span class="w"> </span>--amend
</code></pre></div>
<p>这条命令会直接打开你的终端编辑器让你可以修改提交信息, 也可以组合 <code>reset</code> 命令重新提交:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>git<span class="w"> </span>reset<span class="w"> </span>HEAD^<span class="w"> </span>--soft
$<span class="w"> </span>git<span class="w"> </span>commit<span class="w"> </span>-m<span class="w"> </span><span class="s1">'new message'</span>
</code></pre></div>
<p>这条命令会撤销上一条提交, 并将上一条提交的内容放在工作区内(git add 之后的区域), 然后就可以通过 <code>git commit</code> 重新提交.</p>
<h2 id="_2">合并多个提交</h2>
<p>有时我们为什么了方便记录, 会频繁的创建一些 <strong>临时</strong> 提交, 我们在将提交推送到远端之前可以通过 <code>git rebase</code> 来对现有的多条提交进行合并和编辑:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>git<span class="w"> </span>rebase<span class="w"> </span>-i<span class="w"> </span>origin/master<span class="w"> </span><span class="c1"># master 换成你要推送到远端得分支</span>
<span class="w"> </span><span class="c1"># 基于 Fork -> PR 的流程可以将 origin/master 替换为上游所在的远端分支</span>
</code></pre></div>
<p>这条命令会打开你的终端编辑器, 并显示类似如下的内容:</p>
<div class="highlight"><pre><span></span><code><span class="nt">pick</span><span class="w"> </span><span class="nt">48c9e10</span><span class="w"> </span><span class="nt">tmp</span>
<span class="nt">pick</span><span class="w"> </span><span class="nt">23354c6</span><span class="w"> </span><span class="nt">tmp</span>
<span class="err">#</span><span class="w"> </span><span class="nt">Rebase</span><span class="w"> </span><span class="nt">3b92b89</span><span class="o">.</span><span class="p">.</span><span class="nc">23354c6</span><span class="w"> </span><span class="nt">onto</span><span class="w"> </span><span class="nt">3b92b89</span><span class="w"> </span><span class="o">(</span><span class="nt">2</span><span class="w"> </span><span class="nt">commands</span><span class="o">)</span>
<span class="err">#</span>
<span class="err">#</span><span class="w"> </span><span class="nt">Commands</span><span class="o">:</span>
<span class="err">#</span><span class="w"> </span><span class="nt">p</span><span class="o">,</span><span class="w"> </span><span class="nt">pick</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">use</span><span class="w"> </span><span class="nt">commit</span>
<span class="err">#</span><span class="w"> </span><span class="nt">r</span><span class="o">,</span><span class="w"> </span><span class="nt">reword</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">use</span><span class="w"> </span><span class="nt">commit</span><span class="o">,</span><span class="w"> </span><span class="nt">but</span><span class="w"> </span><span class="nt">edit</span><span class="w"> </span><span class="nt">the</span><span class="w"> </span><span class="nt">commit</span><span class="w"> </span><span class="nt">message</span>
<span class="err">#</span><span class="w"> </span><span class="nt">e</span><span class="o">,</span><span class="w"> </span><span class="nt">edit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">use</span><span class="w"> </span><span class="nt">commit</span><span class="o">,</span><span class="w"> </span><span class="nt">but</span><span class="w"> </span><span class="nt">stop</span><span class="w"> </span><span class="nt">for</span><span class="w"> </span><span class="nt">amending</span>
<span class="err">#</span><span class="w"> </span><span class="nt">s</span><span class="o">,</span><span class="w"> </span><span class="nt">squash</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">use</span><span class="w"> </span><span class="nt">commit</span><span class="o">,</span><span class="w"> </span><span class="nt">but</span><span class="w"> </span><span class="nt">meld</span><span class="w"> </span><span class="nt">into</span><span class="w"> </span><span class="nt">previous</span><span class="w"> </span><span class="nt">commit</span>
<span class="err">#</span><span class="w"> </span><span class="nt">f</span><span class="o">,</span><span class="w"> </span><span class="nt">fixup</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">like</span><span class="w"> </span><span class="s2">"squash"</span><span class="o">,</span><span class="w"> </span><span class="nt">but</span><span class="w"> </span><span class="nt">discard</span><span class="w"> </span><span class="nt">this</span><span class="w"> </span><span class="nt">commit</span><span class="err">'</span><span class="nt">s</span><span class="w"> </span><span class="nt">log</span><span class="w"> </span><span class="nt">message</span>
<span class="err">#</span><span class="w"> </span><span class="nt">x</span><span class="o">,</span><span class="w"> </span><span class="nt">exec</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">run</span><span class="w"> </span><span class="nt">command</span><span class="w"> </span><span class="o">(</span><span class="nt">the</span><span class="w"> </span><span class="nt">rest</span><span class="w"> </span><span class="nt">of</span><span class="w"> </span><span class="nt">the</span><span class="w"> </span><span class="nt">line</span><span class="o">)</span><span class="w"> </span><span class="nt">using</span><span class="w"> </span><span class="nt">shell</span>
<span class="err">#</span><span class="w"> </span><span class="nt">d</span><span class="o">,</span><span class="w"> </span><span class="nt">drop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nt">remove</span><span class="w"> </span><span class="nt">commit</span>
<span class="err">#</span>
<span class="err">#</span><span class="w"> </span><span class="nt">These</span><span class="w"> </span><span class="nt">lines</span><span class="w"> </span><span class="nt">can</span><span class="w"> </span><span class="nt">be</span><span class="w"> </span><span class="nt">re-ordered</span><span class="o">;</span><span class="w"> </span><span class="nt">they</span><span class="w"> </span><span class="nt">are</span><span class="w"> </span><span class="nt">executed</span><span class="w"> </span><span class="nt">from</span><span class="w"> </span><span class="nt">top</span><span class="w"> </span><span class="nt">to</span><span class="w"> </span><span class="nt">bottom</span><span class="o">.</span>
<span class="err">#</span>
<span class="err">#</span><span class="w"> </span><span class="nt">If</span><span class="w"> </span><span class="nt">you</span><span class="w"> </span><span class="nt">remove</span><span class="w"> </span><span class="nt">a</span><span class="w"> </span><span class="nt">line</span><span class="w"> </span><span class="nt">here</span><span class="w"> </span><span class="nt">THAT</span><span class="w"> </span><span class="nt">COMMIT</span><span class="w"> </span><span class="nt">WILL</span><span class="w"> </span><span class="nt">BE</span><span class="w"> </span><span class="nt">LOST</span><span class="o">.</span>
<span class="err">#</span>
<span class="err">#</span><span class="w"> </span><span class="nt">However</span><span class="o">,</span><span class="w"> </span><span class="nt">if</span><span class="w"> </span><span class="nt">you</span><span class="w"> </span><span class="nt">remove</span><span class="w"> </span><span class="nt">everything</span><span class="o">,</span><span class="w"> </span><span class="nt">the</span><span class="w"> </span><span class="nt">rebase</span><span class="w"> </span><span class="nt">will</span><span class="w"> </span><span class="nt">be</span><span class="w"> </span><span class="nt">aborted</span><span class="o">.</span>
<span class="err">#</span>
<span class="err">#</span><span class="w"> </span><span class="nt">Note</span><span class="w"> </span><span class="nt">that</span><span class="w"> </span><span class="nt">empty</span><span class="w"> </span><span class="nt">commits</span><span class="w"> </span><span class="nt">are</span><span class="w"> </span><span class="nt">commented</span><span class="w"> </span><span class="nt">out</span>
</code></pre></div>
<p>内容里面已经说的很明显了, 我们翻译一下关键信息:</p>
<div class="highlight"><pre><span></span><code>#<span class="w"> </span><span class="nv">p</span>,<span class="w"> </span><span class="nv">pick</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>使用此提交
#<span class="w"> </span><span class="nv">r</span>,<span class="w"> </span><span class="nv">reword</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>使用此提交,<span class="w"> </span>但编辑当前提交的提交信息
#<span class="w"> </span><span class="nv">e</span>,<span class="w"> </span><span class="nv">edit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>使用此提交,<span class="w"> </span>但停止修改
#<span class="w"> </span><span class="nv">s</span>,<span class="w"> </span><span class="nv">squash</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>使用此提交,<span class="w"> </span>但将当前提交合并到上一条提交
#<span class="w"> </span><span class="nv">f</span>,<span class="w"> </span><span class="nv">fixup</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>和<span class="w"> </span><span class="nv">squash</span><span class="w"> </span>一样,<span class="w"> </span>但是忽略提交信息
#<span class="w"> </span><span class="nv">x</span>,<span class="w"> </span><span class="k">exec</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>使用<span class="w"> </span><span class="nv">shell</span><span class="w"> </span>运行当前行的命令
#<span class="w"> </span><span class="nv">d</span>,<span class="w"> </span><span class="nv">drop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>移除当前提交
</code></pre></div>
<p>那么我想合并这两条提交并更改一下 <code>commit message</code> 内容如下:</p>
<div class="highlight"><pre><span></span><code>r 48c9e10 一个有意义的提交信息
s 23354c6 tmp # 用 f 也行
</code></pre></div>
<h2 id="_3">总结</h2>
<p>大家可以创建一个练习仓库也体验一下上面命令的不同点, 另外建议多看看 <a href="https://git-scm.com/book/en/v2">Pro Git Book</a>。</p>通过 pyenv 在生产环境安装 Python 32018-11-21T15:00:00+08:002018-11-21T15:00:00+08:00coldtag:www.linuxzen.com,2018-11-21:/tong-guo-pyenv-zai-sheng-chan-huan-jing-an-zhuang-python-3.html<p><a href="https://github.com/pyenv/pyenv">pyenv</a> 是一个简单的 Python 版本管理, 可以安装对应版本的 Python 不依赖系统的包管理, 我用它来在生产和测试环境安装 Python 3.6.</p>
<p>它的 …</p><p><a href="https://github.com/pyenv/pyenv">pyenv</a> 是一个简单的 Python 版本管理, 可以安装对应版本的 Python 不依赖系统的包管理, 我用它来在生产和测试环境安装 Python 3.6.</p>
<p>它的基本原理是安装对应版本的 Python 在它自己的目录下, 然后将对应的 <code>bin</code> 目录通过插入 <code>PATH</code> 变量里实现.</p>
<p>安装可以参考<a href="https://github.com/pyenv/pyenv#basic-github-checkout">官方文档</a>, 但是用它部署 安装在 HOME 目录下会引起一些权限问题, 所以我将安装目录放在了 <code>/srv/pyenv</code> 下:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>git<span class="w"> </span>clone<span class="w"> </span>https://github.com/pyenv/pyenv.git<span class="w"> </span>/srv/pyenv
$<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'export PYENV_ROOT="/srv/pyenv"'</span><span class="w"> </span>>><span class="w"> </span>~/.bash_profile
$<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'export PATH="$PYENV_ROOT/bin:$PATH"'</span><span class="w"> </span>>><span class="w"> </span>~/.bash_profile
$<span class="w"> </span><span class="nb">source</span><span class="w"> </span>~/.bash_profile
</code></pre></div>
<p>上面是针对 Bash shell 的安装, zsh 需将 <code>~/.bash_profile</code> 替换为 <code>~/.zshrc</code></p>
<p>然后就可以使用 <code>pyenv</code> 来安装 Python 了, 我们来安装 Python 3.6.2:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>-y<span class="w"> </span>make<span class="w"> </span>build-essential<span class="w"> </span>libssl-dev<span class="w"> </span>zlib1g-dev<span class="w"> </span>libbz2-dev<span class="w"> </span><span class="se">\</span>
libreadline-dev<span class="w"> </span>libsqlite3-dev<span class="w"> </span>wget<span class="w"> </span>curl<span class="w"> </span>llvm<span class="w"> </span>libncurses5-dev<span class="w"> </span>libncursesw5-dev<span class="w"> </span><span class="se">\</span>
xz-utils<span class="w"> </span>tk-dev
$<span class="w"> </span>pyenv<span class="w"> </span>install<span class="w"> </span><span class="m">3</span>.6.2
</code></pre></div>
<p>安装完成后可以在 Supervisor 等配置文件中通过 <code>/srv/pyenv/versions/3.6.2/bin/python</code> 来使用 Python 3.6.2</p>
<h2 id="_1">加速</h2>
<p>通常情况下 pyenv 下载 Python 包会比较慢,我们可以手动下载相关包上传到比较快的文件服务,然后进行替换。
假设我们上传后得到的 URL 为: https://file.example.com/sources/python/Python-3.6.2.tar.xz ,
可通过如下命令进行替换:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nv">PYENV_PY_VERSION</span><span class="o">=</span><span class="m">3</span>.6.2
$<span class="w"> </span><span class="nv">SPEEDUP_URL</span><span class="o">=</span>https://file.example.com/sources/python/Python-<span class="si">${</span><span class="nv">PYENV_PY_VERSION</span><span class="si">}</span>.tar.xz
$<span class="w"> </span>sed<span class="w"> </span>-i<span class="w"> </span>-e<span class="w"> </span><span class="s2">"s;https://www.python.org/ftp/python/</span><span class="si">${</span><span class="nv">PYENV_PY_VERSION</span><span class="si">}</span><span class="s2">/Python-</span><span class="si">${</span><span class="nv">PYENV_PY_VERSION</span><span class="si">}</span><span class="s2">.tar.xz;</span><span class="si">${</span><span class="nv">SPEEDUP_URL</span><span class="si">}</span><span class="s2">;"</span><span class="w"> </span>/srv/pyenv/plugins/python-build/share/python-build/<span class="si">${</span><span class="nv">PYENV_PY_VERSION</span><span class="si">}</span>
$<span class="w"> </span>pyenv<span class="w"> </span>install<span class="w"> </span><span class="si">${</span><span class="nv">PYENV_PY_VERSION</span><span class="si">}</span>
</code></pre></div>解决 macOS 下安装 pycurl 后导入错误2018-11-20T15:00:00+08:002018-11-20T15:00:00+08:00coldtag:www.linuxzen.com,2018-11-20:/jie-jue-macos-xia-an-zhuang-pycurl-hou-dao-ru-cuo-wu.html<p>在 macOS 下安装 PycURL 后 <code>import curl</code> 会提示:</p>
<div class="highlight"><pre><span></span><code><span class="n">ImportError</span><span class="o">:</span><span class="w"> </span><span class="n">pycurl</span><span class="o">:</span><span class="w"> </span><span class="n">libcurl</span><span class="w"> </span><span class="n">link</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="n">version</span><span class="w"> </span><span class="o">(</span><span class="mf">7.43</span><span class="o">.</span><span class="mi">0</span><span class="o">)</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">older</span><span class="w"> </span><span class="n">than</span><span class="w"> </span><span class="n">compile</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="n">version</span><span class="w"> </span><span class="o">(</span><span class="mf">7.49</span><span class="o">.</span><span class="mi">1</span><span class="o">)</span>
</code></pre></div>
<p>这是因为系统中的 curl 版本过老导致, 可以通过使 …</p><p>在 macOS 下安装 PycURL 后 <code>import curl</code> 会提示:</p>
<div class="highlight"><pre><span></span><code><span class="n">ImportError</span><span class="o">:</span><span class="w"> </span><span class="n">pycurl</span><span class="o">:</span><span class="w"> </span><span class="n">libcurl</span><span class="w"> </span><span class="n">link</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="n">version</span><span class="w"> </span><span class="o">(</span><span class="mf">7.43</span><span class="o">.</span><span class="mi">0</span><span class="o">)</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">older</span><span class="w"> </span><span class="n">than</span><span class="w"> </span><span class="n">compile</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="n">version</span><span class="w"> </span><span class="o">(</span><span class="mf">7.49</span><span class="o">.</span><span class="mi">1</span><span class="o">)</span>
</code></pre></div>
<p>这是因为系统中的 curl 版本过老导致, 可以通过使用 <code>Homebrew</code> 安装最新版来解决:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>brew<span class="w"> </span>install<span class="w"> </span>curl
</code></pre></div>
<p>安装完成后会提示一些信息, 按照提示的信息将 <code>curl</code> 加入到 PATH 路径, 参考:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'export PATH="/usr/local/opt/curl/bin:$PATH"'</span><span class="w"> </span>>><span class="w"> </span>~/.zshrc
</code></pre></div>
<p>如果使用的非 zsh 则将上面替换为:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'export PATH="/usr/local/opt/curl/bin:$PATH"'</span><span class="w"> </span>>><span class="w"> </span>~/.bashrc
</code></pre></div>
<p>完成后重新初始化 shell:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span><span class="nb">source</span><span class="w"> </span>~/.zshrc
</code></pre></div>
<p>然后禁用 <code>pip</code> 缓存重新安装 <code>pycurl</code>:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>pip<span class="w"> </span>uninstall<span class="w"> </span>pycurl
$<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>pycurl<span class="w"> </span>--no-cache-dir
</code></pre></div>
<p>如果使用 <code>tox</code> 可以使用 <code>-r</code> 参数重建 <code>virtualenv</code>:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>tox<span class="w"> </span>-r<span class="w"> </span>-e<span class="w"> </span>py36
</code></pre></div>
<p>若依然存在问题可以手动删除 <code>pip</code> 缓存目录重试:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>rm<span class="w"> </span>-rf<span class="w"> </span>~/Library/Caches/pip/
</code></pre></div>
<p>如遇到下面问题:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nv">__main__</span>.<span class="nv">ConfigurationError</span>:<span class="w"> </span><span class="nv">Curl</span><span class="w"> </span><span class="nv">is</span><span class="w"> </span><span class="nv">configured</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">use</span><span class="w"> </span><span class="nv">SSL</span>,<span class="w"> </span><span class="nv">but</span><span class="w"> </span><span class="nv">we</span><span class="w"> </span><span class="nv">have</span><span class="w"> </span><span class="nv">not</span><span class="w"> </span><span class="nv">been</span><span class="w"> </span><span class="nv">able</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">determine</span><span class="w"> </span><span class="nv">which</span><span class="w"> </span><span class="nv">SSL</span><span class="w"> </span><span class="nv">backend</span><span class="w"> </span><span class="nv">it</span><span class="w"> </span><span class="nv">is</span><span class="w"> </span><span class="nv">using</span>.<span class="w"> </span><span class="nv">Please</span><span class="w"> </span><span class="nv">see</span><span class="w"> </span><span class="nv">PycURL</span><span class="w"> </span><span class="nv">documentation</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nv">how</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">specify</span><span class="w"> </span><span class="nv">the</span><span class="w"> </span><span class="nv">SSL</span><span class="w"> </span><span class="nv">backend</span><span class="w"> </span><span class="nv">manually</span>.
</code></pre></div>
<p>可以通过如下命令重新安装 <code>curl</code></p>
<div class="highlight"><pre><span></span><code>brew<span class="w"> </span>reinstall<span class="w"> </span>curl<span class="w"> </span>--with-openssl
</code></pre></div>
<p>然后重新安装 <code>pycurl</code>,如果还不行可以尝试重启 Shell。</p>
<p>如果通过 <code>tox</code> 依然不行可以设置如下环境变量:</p>
<div class="highlight"><pre><span></span><code><span class="nb">export</span><span class="w"> </span><span class="nv">LDFLAGS</span><span class="o">=</span>-L/usr/local/opt/openssl/lib
<span class="nb">export</span><span class="w"> </span><span class="nv">CPPFLAGS</span><span class="o">=</span>-I/usr/local/opt/openssl/include
</code></pre></div>从单一软件到处处实践 GTD2018-08-05T00:00:00+08:002018-08-05T00:00:00+08:00coldtag:www.linuxzen.com,2018-08-05:/practice-gtd-everywhere.html<blockquote>
<p>这篇文章首发于我们团队内部。</p>
</blockquote>
<p>刚开始接触 <a href="https://zh.wikipedia.org/wiki/%E5%B0%BD%E7%AE%A1%E5%8E%BB%E5%81%9A">GTD</a> 时我是找了一个软件来实践 GTD,
但是效果不是特别好,主要三点:</p>
<ol>
<li>收集麻烦</li>
<li>需 …</li></ol><blockquote>
<p>这篇文章首发于我们团队内部。</p>
</blockquote>
<p>刚开始接触 <a href="https://zh.wikipedia.org/wiki/%E5%B0%BD%E7%AE%A1%E5%8E%BB%E5%81%9A">GTD</a> 时我是找了一个软件来实践 GTD,
但是效果不是特别好,主要三点:</p>
<ol>
<li>收集麻烦</li>
<li>需要单独的软件,我一般会为了节省资源尽量少开软件的的人,所以我对打开很多的软件会很不爽</li>
<li>最主要是懒</li>
</ol>
<p>所以后来我就开始慢慢放弃收集,再也不打开那个软件。再接着我发现其实我们所用到的很多软件都帮我们已经实现了类似的机制。下面举几个我用来实践 GTD 的地方。</p>
<h2 id="_1">邮箱</h2>
<p>大部分情况下如果别人让我做一件事我会让 Ta 给我发一封邮件,如果我没有完成邮件中的工作那这封邮件就会一直在收件箱里躺着,
直到我完成了对应的工作我就会选择将之归档或移动到特定的文件夹,所以我目前有上万封邮件但是收件箱里只有少数没有完成的邮件。</p>
<p>每天我打开邮件看到收件箱我就知道哪些还没有做。</p>
<h2 id="_2">禅道</h2>
<p>我们内部使用禅道做项目管理和 Bug 跟踪,测试和产品会指派给我们一些 Bug 或需求,我们没有完成就会一直在我们名下。
但是有一点是需要注意的就是我们一旦完成(或者说接下来需要他人处理)那么就需要及时变更状态,
这样就可以推动别人来继续。</p>
<p><strong>不要把不属于自己的部分放在自己名下。</strong></p>
<h2 id="gitlab">GitLab</h2>
<p>GitLab 右上角有一个 TODO 的通知,一般我没有完成的我是不会点击完成(Done)的。比如今天有一个发送周报的 Issue 提醒,我会直到我发完周到才会点击 Done 让这个提醒消失。</p>
<h2 id="_3">专门的软件</h2>
<p>我们的工作往往不会是只来源于以上这些渠道,这时候你就需要专门的软件来记录。比如我使用的 Emacs 自带的 Org-mode 来实现 GTD。</p>
<h2 id="_4">浏览器</h2>
<p>我也会用浏览器的标签来记录未完成的任务,比如有些临时任务我没有完成我会保留该浏览器标签。
前提是我有个习惯,就是会及时关闭不再需要的浏览器标签。所以我的浏览器标签一直会处于非常少的状态。</p>
<h2 id="_5">总结</h2>
<p>一旦采用的 GTD 那么最重要的是要时长查看这些地方是否有未完成的工作,并且及时完成。比如我就会经常打开邮件、禅道、GitLab 和我的 Emacs GTD 来查看未完成的任务。</p>
<p>同时如果完成不了或需要他人继续也要及时推动其他人员,这样才能进行高效的工作和协同。</p>创业公司技术进化之路2018-08-04T00:00:00+08:002018-08-04T00:00:00+08:00coldtag:www.linuxzen.com,2018-08-04:/development-eveloping-in-startup.html<p class="first last">本文分享创业公司在技术层面的进化之路。包括技术选型、技术迁移和 CI/CD。</p>
<p>TL;DR</p>
<p>4 年前我有幸加入到现在这家公司,成为了公司早期员工之一,后来在开发上慢慢的得到了一些主动权,然后我就开始凭借着自己的摸索慢慢的完善了一些技术层面相关的东西。
这篇文章主要分享一下这其中的过程,前面的一些文章也有涉及但不全面,这里进行一个总结。其实这篇文章也可以叫作《我在创业公司这 4 年》。</p>
<div class="section" id="section-2">
<h2>面向招聘的技术选型</h2>
<p>开始公司的业务主要是车载物联网硬件,涉及大量 TCP 连接建立和数据传输,所以此时的技术选型主要是 Tornado,主要用于以下两个方面:</p>
<ul class="simple">
<li>TCP Server 处理硬件 TCP 连接</li>
<li>Web 框架,实现客户端 API</li>
</ul>
<p>中间也尝试使用 Flask 实现管理后台,但当时的我对 Tornado 有非常执着的爱,所以在后面的项目中技术选型依然采用 Tornado 作为主要框架。
所以当时的技术选型是:</p>
<ul class="simple">
<li>Tornado 作为 TCP Server 和客户端 API Web 框架</li>
<li>采用 Flask 作为后台管理的框架</li>
<li>前端采用 Bootstrap 3 + jQuery</li>
</ul>
<p>后面公司转型做互联网车险 SaaS 平台也一直沿用这些选型,直到后来业务开始增长需要招人的时候才发现市面上熟悉 Tornado 的人员比较难招,
这时候又面临公司业务拆分,同时当下前后端分离也比较火,后端模板渲染的情况下需要后端了解一些基本的前端知识,而且这种情况下同前端工程师并不太好协同,
所以这时候新的产品线开始采用如下技术选型</p>
<ul class="simple">
<li>Django 作为 Web 框架</li>
<li>前端使用 React</li>
</ul>
<p>选择 React 完全是是出于对 React 社区和 JSX 的喜爱,但是做完一个项目之后发现了一个相同的问题,就是招人非常困难,特别对于小公司来说就更难了。
所以后续就把一些新项目改为采用 Vue,招人也从 React 调整为 Vue。所以就目前来讲最终采用的技术选型就是:</p>
<ul class="simple">
<li>后端使用 Django 作为 Web 框架</li>
<li>前端使用 Vue 作为开发框架</li>
</ul>
<p>从这些就能看出来小公司在技术选型时就不能完全靠喜好,需要结合市面上的人才占比来选择合适的技术。</p>
<p>采用前后端分离架构之后好处是便是对后端的前端相关知识不再是硬性要求,前后端开发可以并行进行,同时前端代码的质量也比较好把控。
我们就出现很多后端工程师连 Bootstrap 都用不好的情况。</p>
<p>但是后端框架使用 Django 好处就是对于人事来说可以更好的筛选简历,因为人事看来简历上出现 Tornado 关键字的实在是太少了,
所以我们当前年度的 3、4 月份招聘旺季算是基本完成招聘任务的。同时也对 Django 有一个全新的认识,纠正了我很多偏见也学到了很多东西。</p>
</div>
<div class="section" id="git">
<h2>基于 Git 的内部包管理</h2>
<p>由于后期的业务转型的时候项目拆分比较细,所以从这时候起就开始有意识的提取一些公共逻辑放在单独的包里,
早期这个包是通过拷贝的方式放在不同的项目里,这样更新就比较麻烦,后续了解到 <tt class="docutils literal">pip</tt> 是可以通过 <cite>git</cite> 来安装包的,
所以后来就将这个包放在单独的仓库,每个项目通过在 <tt class="docutils literal">requirements.txt</tt> 增加类似下面的内容来引用:</p>
<div class="highlight"><pre><span></span>git+ssh://git@github.com/Owner/foobar@v2.14
</pre></div>
<p>这样有好处也有一些坏处,好处就是更新迭代方便,坏处就是增加部署成本。</p>
</div>
<div class="section" id="section-3">
<h2>部署</h2>
<div class="section" id="fabric">
<h3>Fabric</h3>
<p>早期项目只有一个所以部署基本是手动,后期项目开始变得多起来,手动部署就变得麻烦起来,这时候开始使用 Fabric 写一些部署脚本。
后来就写了一套通用的规则,然后暴漏出一些配置接口,配置诸如仓库 URL,Supervisor 启动项等等,然后将之放在通用库里,
同时编写一套 <tt class="docutils literal">Makefile</tt> 的规则放在通用库里,我们的通用库取名是 <tt class="docutils literal">allspark</tt>。</p>
<p>这样直接在项目中放一个 <tt class="docutils literal">Makefile</tt> 添加类似下面的内容:</p>
<div class="highlight"><pre><span></span><span class="nv">ALLSPARK_PTH</span><span class="o">=</span><span class="k">$(</span>shell<span class="w"> </span>python<span class="w"> </span>-c<span class="w"> </span><span class="s1">'import os; import allspark;print(os.path.dirname(allspark.__file__))'</span><span class="k">)</span>
<span class="cp">include $(ALLSPARK_PTH)/mkrules.mk</span>
</pre></div>
<p>这样就可以通过 <tt class="docutils literal">make</tt> 进行一键部署,这套机制用了很长时间,目前还有部分维护较少的项目还再使用。</p>
</div>
<div class="section" id="ansible">
<h3>Ansible</h3>
<p>后来开始立项 Django 的项目,由于 <tt class="docutils literal">fabric</tt> 相关的封装都在 Tornado 的通用库里,再继续使用就比较麻烦,这时候我就开始寻求新的解决方案,
所以我又开始研究 Ansible,经过简单的了解和尝试我发现只要定义几个简单 role 就可以满足需求,所以我新建了一个仓库存放这些角色,
Ansible 提供了 <tt class="docutils literal"><span class="pre">ansible-galaxy</span></tt> 来安装依赖,所以再每个项目下都有如下 Ansible 配置文件:</p>
<ul>
<li><p class="first">ansible.cfg -- 指定 hosts 位置</p>
</li>
<li><p class="first">deploy/ansible/hosts -- 配置主机</p>
</li>
<li><p class="first">deploy/ansible/requirements.yml -- 指定依赖</p>
<p>内容如下</p>
<div class="highlight"><pre><span></span>- src: git@gitlab.example.com:username/ansible-roles
scm: git
name: ansible-roles
</pre></div>
</li>
<li><p class="first">deploy/ansible/dev-playbook.yml -- 测试环境部署规则,指定变量和角色实现部署</p>
</li>
<li><p class="first">deploy/ansible/prod-playbook.yml -- 生产环境部署,指定变量和角色实现部署</p>
</li>
</ul>
<p>现在部署相关的就都开始使用 Ansible,部署命令如下:</p>
<div class="highlight"><pre><span></span><span class="c1"># 需要在控制机上安装依赖</span>
$<span class="w"> </span>ansible-galaxy<span class="w"> </span>install<span class="w"> </span>-r<span class="w"> </span>deploy/ansible/requirements.yml
$<span class="w"> </span>ansible-playbook<span class="w"> </span>deploy/ansible/dev-playbook.yml
</pre></div>
<p>这里 ansible-galaxy 有个坑,就是不支持更新,要想更新已安装的 ansible 角色需要手动删除并重新安装:</p>
<div class="highlight"><pre><span></span>$<span class="w"> </span>rm<span class="w"> </span>~/.ansible/roles/ansible-roles
$<span class="w"> </span>ansible-galaxy<span class="w"> </span>install<span class="w"> </span>-r<span class="w"> </span>deploy/ansible/requirements.yml
</pre></div>
</div>
<div class="section" id="deploy-key-ssh-agent-forwarding">
<h3>Deploy Key 到 SSH agent forwarding</h3>
<p>在部署的过程中需要在服务器上拉取代码,就涉及到仓库权限的问题我们一开始的解决办法是部署之前通过 <tt class="docutils literal">Fabric</tt> 上传一个 Deploy Key
到目标服务器,在部署完成之后再将对应的 Deploy Key 删除。</p>
<p>一个 Deploy Key 只对应一个仓库的只读权限,这种模式在前期是没有问题的,但是到了后期我们把通用库拆分到独立的仓库通过 <tt class="docutils literal">pip</tt> 进行安装时就遇到了问题,
这个时候我们开始抛弃 Deploy Key 改为使用 SSH agent forwarding,具体请参见 <a class="reference external" href="https://developer.github.com/v3/guides/using-ssh-agent-forwarding/">Using SSH agent forwarding</a> 。</p>
</div>
</div>
<div class="section" id="orm">
<h2>ORM 使用</h2>
<div class="section" id="sql">
<h3>手写 SQL</h3>
<p>早期我们使用手写 SQL 的方式与数据库交互,慢慢的我们发现这种方式存在一些问题:</p>
<ul class="simple">
<li>大块 SQL 语句在代码中异常丑陋</li>
<li>非常容易编写错误的 SQL 语句</li>
<li>代码 Review 过程中要额外注意 SQL 注入相关问题</li>
<li>为了防止注入,根据条件拼接 SQL 语句比较困难同时拼接代码看起来丑陋并且难以理解</li>
</ul>
</div>
<div class="section" id="sqlalchemy-core">
<h3>SQLAlchemy Core</h3>
<p>后来发现 SQLAlchemy 对外提供的接口是分为两层的:</p>
<ul class="simple">
<li>Core -- 语句生成引擎</li>
<li>ORM -- 基于语句生成引擎的 ORM</li>
</ul>
<p>发现只使用 Core 和手写 SQL 并无太大区别,但是解决了上面的所有问题,请看下面示例:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span>
<span class="kn">from</span> <span class="nn">.</span> <span class="kn">import</span> <span class="n">db_engine</span>
<span class="kn">from</span> <span class="nn">.</span> <span class="kn">import</span> <span class="n">Table</span>
<span class="k">with</span> <span class="n">db_engine</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">db</span><span class="p">:</span>
<span class="n">db</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
<span class="n">Table</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
<span class="o">.</span><span class="n">where</span><span class="p">(</span>
<span class="p">(</span><span class="n">Table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="o">&</span>
<span class="p">(</span><span class="n">Table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">sex</span> <span class="o">==</span> <span class="s2">"male"</span><span class="p">)</span>
<span class="o">&</span>
<span class="n">Table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">is_valid</span>
<span class="p">)</span>
<span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">sa</span><span class="o">.</span><span class="n">desc</span><span class="p">(</span><span class="n">Table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">))</span>
<span class="p">)</span>
</pre></div>
<p>同时也可以支持复杂的 SQL 语句,具体请参见 <a class="reference external" href="https://docs.sqlalchemy.org/en/latest/core/tutorial.html">文档</a> 。</p>
</div>
<div class="section" id="django-orm">
<h3>Django ORM</h3>
<p>再后来的新项目都采用了 Django 并使用 Django 自带的 ORM。</p>
</div>
</div>
<div class="section" id="section-4">
<h2>本地开发</h2>
<p>我们应用的服务依赖较少,目前只依赖 MySQL 和 Redis。一开始大家都通过统一连接内网的同一服务进行本地开发,
这种开发模式会带来一个问题:</p>
<ol class="arabic simple">
<li>假设其中一名开发人员删除了一个字段,并调整了对应的代码但是没有提交</li>
<li>由于我们基本上是 TDD 的开发模式,此时就会导致其他人的单元测试无法正常运行</li>
</ol>
<p>基于这种模式我们本地开发引入 Docker,使用 Docker 在本地启动 MySQL 和 Redis 服务,在我们的通用库里提供以下两个文件:</p>
<ul>
<li><p class="first">docker-compose.yml</p>
<div class="highlight"><pre><span></span>version: '3'
services:
mysql:
image: mysql:5.6
restart: always
ports:
- "3306:3306"
volumes:
- ~/.botpy/etc/mysql/conf.d:/etc/mysql/conf.d
- ~/.botpy/data/mysql:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root-password-you-should-replace
redis:
image: redis
restart: always
ports:
- "6379:6379"
</pre></div>
</li>
<li><p class="first">init-db.py -- 包装 mysqldump 和 mysql 命令实现同步内网数据库的脚本</p>
</li>
</ul>
<p>数据库迁移最终流程就变为:</p>
<ol class="arabic simple">
<li>在本地编写并调试,并先应用到本地</li>
<li>提交 MR 在内网的测试数据库进行验证</li>
<li>验证通过后合并 MR 触发部署<ol class="arabic">
<li>迁移应用到内网的开发数据库</li>
<li>部署到测试数据库</li>
</ol>
</li>
</ol>
</div>
<div class="section" id="section-5">
<h2>数据库迁移</h2>
<div class="section" id="sqlalchemy-migrate">
<h3>SQLAlchemy Migrate</h3>
<p>早期我们是通过手动收集变更的 SQL 语句到指定文件下,然后在上线之前手动在数据库进行执行。后来这块就导致了很多上线问题,
主要是忘记收集和忘记执行,后面在找这方面的解决方案时发现了 <a class="reference external" href="http://sqlalchemy-migrate.readthedocs.io/en/latest/">SQLAlchemy Migrate</a> ,经过简单的改造之后用起来还算舒心。</p>
<p>我们在用的过程中遇到的主要问题中文编码问题,经过排查是由于 sqlparse 库导致的,由于没有暴漏相关接口加上 <a class="reference external" href="http://sqlalchemy-migrate.readthedocs.io/en/latest/">SQLAlchemy Migrate</a>
是由 OpenStack 维护,想要贡献代码非常困难,所以就通过在 <tt class="docutils literal">manage.py</tt> 增加如下代码来解决:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">functools</span>
<span class="kn">import</span> <span class="nn">sqlparse</span>
<span class="c1"># HACK!!: 替换原函数修复 sqlalchemy-migrate 的编码 bug</span>
<span class="n">sqlparse</span><span class="o">.</span><span class="n">format</span> <span class="o">=</span> <span class="n">functools</span><span class="o">.</span><span class="n">partial</span><span class="p">(</span><span class="n">sqlparse</span><span class="o">.</span><span class="n">format</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="s2">"utf8"</span><span class="p">)</span>
</pre></div>
</div>
<div class="section" id="alembic">
<h3>Alembic</h3>
<p>虽然 <a class="reference external" href="http://sqlalchemy-migrate.readthedocs.io/en/latest/">SQLAlchemy Migrate</a> 已经满足需求,但是在用的过程中发现会有两个问题:</p>
<ol class="arabic simple">
<li>社区更新不积极,</li>
<li>由于 <a class="reference external" href="http://sqlalchemy-migrate.readthedocs.io/en/latest/">SQLAlchemy Migrate</a> 仅记录最后一个版本号,不便于多人开发,考虑如下场景:<ol class="arabic">
<li>A 增加了版本 001-foo</li>
<li>B 没有拉取代码也进行迁移就会也增加一个 001-bar</li>
<li>A 的代码合并并部署,001-foo 会被执行(数据库标记 001 已执行)</li>
<li>B 的代码合并并部署,由于 001 被标记为已执行 001-bar 不会进行执行</li>
</ol>
</li>
</ol>
<p>这时候发现 SQLAlchemy 的作者推出了 <a class="reference external" href="http://alembic.zzzcomputing.com/en/latest/">Alembic</a>,经过简单的尝试发现该工具更加强大支持类似 Git 的版本控制,
开发也比较活跃,解决了上面两个问题。所以在后续就使用 <a class="reference external" href="http://alembic.zzzcomputing.com/en/latest/">Alembic</a> 替换了 <a class="reference external" href="http://sqlalchemy-migrate.readthedocs.io/en/latest/">SQLAlchemy Migrate</a> 。</p>
</div>
<div class="section" id="django-migrate">
<h3>Django Migrate</h3>
<p>后续 Django 的新项目就开始采用 Django 自带的数据库迁移方案,这里就不再细述。</p>
</div>
</div>
<div class="section" id="section-6">
<h2>单元测试</h2>
<div class="section" id="section-7">
<h3>测试数据库</h3>
<p>我觉的我们之所以能成功的推行了单元测试,并将之作为日常开发中衡量代码的标准之一,最大的功劳就是解决了测试数据库相关问题,
就像我之前的文章提到的:</p>
<blockquote>
很长一段时间以来写单元测试都类似写执行脚本,运行一下然后看一下结果。</blockquote>
<p>很大的原因就是没有解决测试数据库相关的问题,比如我写了一个测试然后在我本地数据库填充了数据,测试通过了。然后后面数据再变动测试就失败了。
为了解决这个问题我自己首先实现了一个基于 unittest 的测试收集和运行器,然后在测试运行开始之前插入一段代码做如下事情:</p>
<ol class="arabic simple">
<li>连接配置文件中的数据库并读取表结构信息</li>
<li>根据一定规则生成创建一个新的数据库</li>
<li>将读取的表结构信息应用到新数据库</li>
<li>加载测试包下的一些 SQL 文件并在新的数据库中执行</li>
<li>Patch 配置文件,将数据库名调整到新的数据库</li>
</ol>
<p>后面我们切换到 pytest 作为 test runner 后将这一块逻辑封装成了一个 pytest 的插件。加上后面上的 mock 我们的单元测试才真正的完善。
这也造就了我们大部分项目都达到了 80% ~ 95% 的单元测试覆盖率,同时也为我们之后迁移 Python 3 打下了很好的基础。</p>
</div>
<div class="section" id="pytest-test-runner">
<h3>pytest 作为 test runner</h3>
<p>前面也提到了,一开始我们用的是基于 unittest 自己实现的测试发现、收集和运行的工具,这一块也是我受之前一家公司的影响,
后来发现 pytest 除了是一个强大的测试框架,同时也可以单独用来作为 test runner 使用,我不太喜欢 pytest 这种函数式方式
编写测试,很多 fixture markup 感觉太过隐式,所以我只拿 pytest 作为 test runner,单元测试还是使用 unittest 那一套。</p>
<p>pytest 作为 test runner 对比 unittest 的好处是:</p>
<ul class="simple">
<li>社区活跃</li>
<li>生态好,插件多</li>
<li>结合插件系统非常容易扩展和自定义</li>
</ul>
</div>
<div class="section" id="mock">
<h3>mock</h3>
<p>参见之前的 <a class="reference external" href="/python-mock-shi-yong-xin-de.html">博文</a> 。</p>
</div>
<div class="section" id="tox">
<h3>tox</h3>
<p>开始每次运行测试构建测试相关的 Python 环境会比较麻烦,后面接触了 <a class="reference external" href="http://tox.readthedocs.io/en/latest/">tox</a> 可以很方便的构建测试环境,
同时支持多个 Python 版本环境构建。</p>
</div>
</div>
<div class="section" id="ci-cd">
<h2>CI/CD</h2>
<div class="section" id="bors-homu">
<h3>结识 bors 和 homu</h3>
<p>有一段时间我特别关注 Rust 社区,看到他们有一个机器人专门跑单元测试和合并 PR,我就开始思考能不能用到我们的项目中,
经过简单的观察我首先接触到了 <a class="reference external" href="https://github.com/graydon/bors">bors</a> ,初次使用后感觉真心酷炫,但是由于它是采用轮询会有如下两个问题:</p>
<ol class="arabic simple">
<li>时效性不好</li>
<li>容易达到 GitHub 接口请求次数上限</li>
<li>还有其他一些功能上的不全</li>
</ol>
<p>这时候我就发现 Rust 社区早一不使用 <a class="reference external" href="https://github.com/graydon/bors">bors</a> 而是改为使用 <a class="reference external" href="https://github.com/barosl/homu">homu</a> 了,具体的信息可以参见之前的一篇
<a class="reference external" href="/python-github-si-you-xiang-mu-tong-guo-buildbot-jin-xing-review.html">文章</a> 。</p>
</div>
<div class="section" id="github">
<h3>GitHub</h3>
<p>参见 <a class="reference external" href="/python-github-si-you-xiang-mu-tong-guo-buildbot-jin-xing-review.html">Python github 私有项目通过 buildbot 进行 Review</a> 。
后面也尝试迁移到 buildbot 0.9,具体方案参见 <a class="reference external" href="https://github.com/servo/homu/pull/119">Add buildbot 0.9 support steps in README.md #119</a> 。</p>
</div>
<div class="section" id="gitlab">
<h3>GitLab</h3>
<div class="section" id="pipelines">
<h4>Pipelines</h4>
<p>迁移到 GitLab 之后就弃用了 buildbot 改用 GitLab 自身的那一套 CI/CD。具体参见 <a class="reference external" href="https://about.gitlab.com/features/gitlab-ci-cd/">GitLab CI/CD</a> 。</p>
<p>我们 CI 构建用的自己构建的 Docker 镜像,里面集成了一些基础依赖包和部署需要的相关信息。</p>
</div>
<div class="section" id="homu-gitlab">
<h4>homu-gitlab</h4>
<p>由于业务扩张 GitHub 上的私有仓库成本开始提升,所以就开始考虑迁移自建的 GitLab,这时面临一个问题就是怎么保持现有的工作流不变,
首先就是要找到一个 <a class="reference external" href="https://github.com/barosl/homu">homu</a> 的 GitLab 版实现,经过自己的搜寻后并没有发现合适的,所以我就尝试 Fork 了一份尝试自己迁移到 GitLab,
最终成功的迁移并应用到 GitLab 中来,参见 <a class="reference external" href="https://github.com/coldnight/homu-gitlab">coldnight/homu-gitlab</a> 。</p>
<p>一些不完美的地方</p>
<ol class="arabic">
<li><p class="first">需要依赖 SSH 私玥</p>
<p>由于 GitLab 的接口并没有 GitHub 那么完善,没有相关合并 MR 的接口,所以将之前主要依赖 GitHub 接口的部分都统一改成了通过 git 命令操作。</p>
</li>
<li><p class="first">同步功能未实现,每次同步需要通过重启实现</p>
</li>
</ol>
</div>
</div>
<div class="section" id="section-8">
<h3>持续进化</h3>
<p>在上面完成之后最近还做了一些调整和优化:</p>
<ol class="arabic simple">
<li>启用 GitLab CI/CD 缓存来加速和优化 CI 构建</li>
<li>内网搭建 Docker Registry 统一托管和构建 CI Docker 镜像(这部分也是自动化的)</li>
<li>通过一个仓库托管 homu 的配置文件,每次调整提交后自动重启 homu 服务</li>
</ol>
</div>
<div class="section" id="pre-commit">
<h3>pre-commit</h3>
<p>我们一开始代码检测主要利用 Git 的 pre-commit 自己编写 shell 脚本来组合,后来发现 <a class="reference external" href="https://github.com/pre-commit/pre-commit">pre-commit</a> 后开始统一替换为 <a class="reference external" href="https://github.com/pre-commit/pre-commit">pre-commit</a> 。
<a class="reference external" href="https://github.com/pre-commit/pre-commit">pre-commit</a> 的作者也推出了其他很多代码检测相关的工具,我也应用到了我们自己的项目上。</p>
</div>
</div>
<div class="section" id="python">
<h2>Python</h2>
<p>参见 <a class="reference external" href="/qian-yi-dao-python-3.html">迁移到 Python 3</a> 。</p>
</div>
<div class="section" id="section-9">
<h2>总结</h2>
<p>简单总结一下目前整个技术栈:</p>
<ul class="simple">
<li>Python 3<ul>
<li>Django/Tornado</li>
<li>SQLAlchemy</li>
<li>pytest</li>
<li>tox</li>
<li>Alembic</li>
<li>Celery</li>
</ul>
</li>
<li>Ansible</li>
<li>GitLab<ul>
<li>CI/CD</li>
<li>homu-gitlab</li>
<li>pre-commit</li>
</ul>
</li>
<li>Docker<ul>
<li>docker-compose</li>
</ul>
</li>
<li>Vue</li>
<li>Sentry</li>
</ul>
</div>
迁移到 Python 32017-07-13T00:00:00+08:002017-07-13T00:00:00+08:00coldtag:www.linuxzen.com,2017-07-13:/qian-yi-dao-python-3.html<p>前段时间(2017-06-07)我开始决定将公司现有的项目逐渐的迁移到 Python 3. 主要原因有一下几点:</p>
<ul>
<li>
<p>Python 3.6 新增了一些新的特性我很喜 …</p></li></ul><p>前段时间(2017-06-07)我开始决定将公司现有的项目逐渐的迁移到 Python 3. 主要原因有一下几点:</p>
<ul>
<li>
<p>Python 3.6 新增了一些新的特性我很喜欢</p>
<p>包括:</p>
<ul>
<li><a href="https://www.python.org/dev/peps/pep-0484/">PEP484 类型注解</a></li>
<li><a href="https://www.python.org/dev/peps/pep-0492/">PEP492 原生的协程异步: async and await</a></li>
<li><a href="https://www.python.org/dev/peps/pep-0498/">PEP498 格式化字符串</a></li>
<li>Python 3 的生态已经完善, 我们所使用的一些第三方库都已支持 Python 3(或有其他成熟的替代)</li>
<li>Python 2 到 2020 年就不在维护</li>
</ul>
</li>
</ul>
<p><strong>促成我决定迁移到 Python 3 的主要原因是公司最大的项目的单元测试覆盖率经过一段时间的迭代终于达到了 80% 以上.</strong></p>
<h2 id="_1">迁移方案</h2>
<p>由于项目巨大任务艰巨无法短时间内就将项目迁移到 Python 3, 而且当前还有产品上的功能需要迭代.
所以迁移方案是同时兼容 Python 2 和 3, 并在迁移完成之后移除对 Python 2 的支持.</p>
<h2 id="ci">搭建 CI</h2>
<p>由于之前本身就有持续集成服务(参见<a href="https://www.linuxzen.com/python-github-si-you-xiang-mu-tong-guo-buildbot-jin-xing-review.html">上一篇</a>), 所以目前需要做的就是在现有的基础增加一套针对 Python 3 的持续集成服务.
并且在没有完全兼容 Python 3 之前针对 Python 3 的持续集成不影响最终构建结果, 仅作为参考.
这里根据不同的 CI 需要进行不同的操作不在本文讨论范围之内就不在深入.</p>
<h2 id="_2">兼容库的选择</h2>
<p>针对一些 Python 2 和 3 的差异我不打算自己封装, 所以找了一些成熟的第三方库, 我对比了目前比较流行的两个兼容库:</p>
<ul>
<li><a href="http://python-future.org/">future</a></li>
<li><a href="https://pythonhosted.org/six/">six</a></li>
</ul>
<p>future 看起来使用比较多的黑魔法, 好多兼容都是侵入式的修改. 所以我选择了更加清晰的 six.</p>
<h2 id="_3">兼容性问题及解决</h2>
<p>可以从 <code>__future__</code> 导入如下特性来解决一些常见的不兼容:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">print_function</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">division</span> <span class="c1"># 解决除法行为不一致</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">unicode_literals</span> <span class="c1"># 字符串字面量默认为 Unicode</span>
</code></pre></div>
<h3 id="_4">标准库和内建函数不兼容</h3>
<p>Python 3 中更改了一些标准库的名字和一些函数归属的模块, 同时也移除了一些关键字如 <code>round</code>.
此种类型的问题可以通过 <code>six.moves</code> 来进行解决:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">six.moves.urllib_parse</span> <span class="k">as</span> <span class="n">urlparse</span>
<span class="kn">from</span> <span class="nn">fix.moves.builtins</span> <span class="kn">import</span> <span class="nb">round</span>
</code></pre></div>
<h3 id="_5">字符串</h3>
<p>以我的感受解决 Python 2 和 3 之间的兼容性字符串是难度最大的问题之一, 因为 Python 3 重新实现了字符串相关,
并增加了一些限制, 主要体现在:</p>
<ol>
<li>移除了 unicode 关键字</li>
<li>原有的 str(bytes) 变成了 unicode</li>
<li>bytes 不在支持 format/decode 等方法</li>
<li>不支持 Unicode 和字节序直接拼接和对比(<code>in</code> 操作), 一些标准库也明确指定了接收的是字节序还是 Unicode.</li>
<li>通过字符串格式化拼接 Unicode 和字节序会产生不符合预期的结果</li>
</ol>
<p>这里面第 <code>1</code> 条和第 <code>2</code> 条是比较容易, 可以通过批量的查找和替换来解决. 3-4 是比较难解决, 但是单元测试基本可以覆盖到.</p>
<p>5 会产生一些非常难以发现的 Bug, 因为出现像下面这样的结果:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="s1">'</span><span class="si">{}</span><span class="s1">'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="sa">b</span><span class="s1">'test'</span><span class="p">)</span>
<span class="s2">"b'test'"</span> <span class="c1"># Python 2: "test"</span>
</code></pre></div>
<p>如果一些单元测试没覆盖到的或者本身逻辑就是被 mock 掉就更难排查.
比如我们内部服务通信大量的使用了签名机制, 在进行单元测试时又对这部分逻辑进行了 mock,
所以在我们在测试环境用 Python 3 启动项目后主要就是解决此类问题.</p>
<h3 id="_6">迭代器</h3>
<p>Python 3 中除了字符串这一改动难以兼容, 还有一个就是之前返回列表的一些函数或方法改为返回迭代器, 如</p>
<ul>
<li>dict.keys/dict.values/dict.items</li>
<li>map/filter/zip</li>
</ul>
<p>如果没有对一些迭代器进行展开而是当做列表使用就会产生异常, 或者对迭代器展开没有及时进行收集就会产生一些难以排查的 Bug,
考虑如下代码:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">():</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"a"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"n"</span><span class="p">:</span> <span class="s2">"1"</span><span class="p">}</span> <span class="s2">"b"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"n"</span><span class="p">:</span> <span class="s2">"2"</span><span class="p">}}</span>
<span class="n">digits</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">values</span><span class="p">()</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">digits</span><span class="p">:</span>
<span class="n">item</span><span class="p">[</span><span class="s2">"n"</span><span class="p">]</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">item</span><span class="p">[</span><span class="s2">"n"</span><span class="p">])</span>
<span class="k">return</span> <span class="n">digits</span>
</code></pre></div>
<p>上面代码在 Python 2 中可以达到预期的行为, 返回 <code>[{"n": 1}, {"n": 2}]</code>, 但在 Python 3 会返回一个消耗完的迭代器, 转换之后结果为 <code>[]</code>.</p>
<p>下面代码在 Python2 是可以正常工作的, 但是 Python 3 下不行:</p>
<div class="highlight"><pre><span></span><code><span class="n">digits</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"a"</span><span class="p">:</span> <span class="s2">"1"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">:</span> <span class="s2">"2"</span><span class="p">}</span>
<span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">digits</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
<span class="n">digits</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">digits</span><span class="p">[</span><span class="n">k</span><span class="p">])</span>
</code></pre></div>
<p>主要是 Python 2 该方法返回一个列表是对字典 key 的一次拷贝, 所以在更改字典原有值就不会有问题,
但是 Python 3 中该方法返回了一个迭代器, 是对字典 key 的引用. 如果这时候更新字典就会触发异常.</p>
<h2 id="_7">工具</h2>
<h3 id="pylint">pylint</h3>
<p><code>pylint</code> 有一个选项 <code>--py3k</code> 选项可以打开检测一些不兼容 Python 3 的地方, 比如 <code>map/filter/zip</code> 没有展开等.</p>
<h3 id="pre-commit">pre-commit</h3>
<p>准确的说应该是 <code>pre-commit</code> 下有一些工具可以帮助我们自动修复一些 Python 3 的兼容性问题,
并且可以方便的放在持续集成服务上运行监测, 这里列出一些我们用于帮助 Python 3 迁移的工具:</p>
<ul>
<li><a href="https://github.com/asottile/reorder_python_imports">reorder_python_imports</a> 主要可以通过 <code>--add-import</code> 为每次修改的文件增加从 <code>__future__</code> 模块引入一些特性.</li>
<li><a href="https://github.com/asottile/pyupgrade">pyupgrade</a> 在提交时自动修正一些不兼容的地方</li>
<li>我对 pylint 简单的包装<a href="https://github.com/coldnight/pre-commit-pylint">pre-commit-pylint</a></li>
</ul>
<p>最后贴上我们项目所使用的 <code>.pre-commit-config</code> 文件</p>
<div class="highlight"><pre><span></span><code><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repo</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git@github.com:pre-commit/pre-commit-hooks.git</span>
<span class="w"> </span><span class="nt">sha</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v0.8.0</span>
<span class="w"> </span><span class="nt">hooks</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">flake8</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">check-docstring-first</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">debug-statements</span>
<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repo</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">https://github.com/asottile/reorder_python_imports</span>
<span class="w"> </span><span class="nt">sha</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v0.3.5</span>
<span class="w"> </span><span class="nt">hooks</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">reorder-python-imports</span>
<span class="w"> </span><span class="nt">language_version</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">python2.7</span>
<span class="w"> </span><span class="nt">args</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--separate-relative</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--separate-from-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--add-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">from __future__ import absolute_import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--add-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">from __future__ import division</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--add-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">from __future__ import print_function</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--add-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">from __future__ import unicode_literals</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--remove-import</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">from __future__ import with_statement</span>
<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repo</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">https://github.com/asottile/pyupgrade</span>
<span class="w"> </span><span class="nt">sha</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v1.1.1</span>
<span class="w"> </span><span class="nt">hooks</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pyupgrade</span>
<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repo</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git@github.com:coldnight/pre-commit-pylint.git</span>
<span class="w"> </span><span class="nt">sha</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">630e2662aabf3236fc62460b163d613c4bd1cfbc</span>
<span class="w"> </span><span class="nt">hooks</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pylint-py3k</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">id</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pylint-score-limit</span>
<span class="w"> </span><span class="nt">args</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--limit=8.5</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">--rcfile=./.pylintrc</span>
<span class="w"> </span><span class="nt">additional_dependencies</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">enum34; python_version<='3.4'</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mock</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">coverage</span>
</code></pre></div>
<h2 id="_8">后记</h2>
<p>目前我们已经完成了公司最大的一个项目的 Python 3 兼容, 并于今天在测试环境使用 Python 3 运行.</p>
<h2 id="_9">参考</h2>
<ul>
<li><a href="https://docs.python.org/3/howto/pyporting.html">官方指南</a></li>
<li><a href="http://www.zlovezl.cn/articles/instagram-pycon-2017/">Instagram 在 PyCon 2017 的演讲摘要</a></li>
</ul>Python github 私有项目通过 buildbot 进行 Review2016-05-22T00:00:00+08:002016-05-22T00:00:00+08:00coldtag:www.linuxzen.com,2016-05-22:/python-github-si-you-xiang-mu-tong-guo-buildbot-jin-xing-review.html<h2 id="_1">背景</h2>
<p>随着公司开发团队的壮大, 团队中每个人的水平参差不齐, 为了保证项目质量我们打算对
提交的代码进行 review, 但是苦 …</p><h2 id="_1">背景</h2>
<p>随着公司开发团队的壮大, 团队中每个人的水平参差不齐, 为了保证项目质量我们打算对
提交的代码进行 review, 但是苦于一直没有好的 review 机制. 前段时间我在逛
<a href="https://github.com/rust-lang/rust">Rust</a> 社区是发现了他们有一个 review 机器人 <a href="https://github.com/barosl/homu">Homu</a> 非常不错,
研究一下后我将其应用到我们当前 Python 项目中来配合 review, 我感觉非常棒,
今天抽空就分享给大家.</p>
<h2 id="_2">技术栈</h2>
<p>本文涉及的项目和技术有:</p>
<ul>
<li><a href="https://github.com/barosl/homu">Homu</a></li>
<li><a href="https://github.com/buildbot/buildbot">buildbot</a></li>
<li><a href="https://github.com/sebdah/git-pylint-commit-hook">git-pylint-commit-hook</a></li>
<li><a href="https://www.docker.com">Docker</a></li>
</ul>
<h2 id="0-github-key">0. 隔离 Github 部署 Key</h2>
<p>Github 可以添加部署 Key 来实现部署, 但是每个项目必须是不同的部署 key. 这就给
多个私有项目的可持续集成带来一定的困难, 因为 buildbot 是通过轮询来获取 git 分支
变更的, 并且 buildbot 不支持指定私钥. </p>
<p>我们前期想到的是通过 <a href="https://www.docker.com">Docker</a> 来分别跑每个项目的 buildbot, 这样部署 key 就
可以很容易进行隔离. 后面我们找到一种更好的方式来指定部署 Key: 通过 <code>ssh_config</code>
来指定别名.</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cat<span class="w"> </span>~/.ssh/config
Host<span class="w"> </span>github-project-a-alias
<span class="w"> </span>Hostname<span class="w"> </span>github.com
<span class="w"> </span>IdentityFile<span class="w"> </span><span class="s2">"~/.ssh/project_a_deploy_key"</span>
Host<span class="w"> </span>github-project-b-alias
<span class="w"> </span>Hostname<span class="w"> </span>github.com
<span class="w"> </span>IdentityFile<span class="w"> </span><span class="s2">"~/.ssh/project_b_deploy_key"</span>
</code></pre></div>
<p>这样就可以将原有的项目地址做如下转换</p>
<div class="highlight"><pre><span></span><code><span class="gd">- git@github.com:Owner/a</span>
<span class="gi">+ git@github-project-a-alias:Owner/a</span>
<span class="gd">- git@github.com:Owner/b</span>
<span class="gi">+ git@github-project-b-alias:Owner/b</span>
</code></pre></div>
<h2 id="1-homu">1. 搭建 homu</h2>
<p><a href="https://github.com/barosl/homu">Homu</a> 是一个构建在 Github 和 buildbot(或 Travis) 之上的工具, 它通过监听
PR 里的评论来触发 buildbot 构建, 并监听构建结果, 如果构建成功则自动合并 PR</p>
<p><a href="https://github.com/barosl/homu">Homu</a> 的搭建可以参见其项目地址有简单的介绍, 非常简单这里就不在赘述.</p>
<p>Homu 是针对 Rust 开发, 并没有考虑私有项目, 所以其信息都是公开的, 为了减少
暴漏信息带来的安全性问题, 我们将 Homu 搭建在内网并通过 ssh 端口转发将 Homu 的
端口发送到一台公网服务器:</p>
<div class="highlight"><pre><span></span><code>ssh<span class="w"> </span>-o<span class="w"> </span><span class="nv">TCPKeepAlive</span><span class="o">=</span>yes<span class="w"> </span>-CfNgR<span class="w"> </span><span class="m">127</span>.0.0.1:54856:127.0.0.1:54856<span class="w"> </span>user@remote.ip.address
</code></pre></div>
<p>然后通过以下 nginx 配置文件来仅暴漏供 Github webhook 调用的地址:</p>
<div class="highlight"><pre><span></span><code><span class="nt">upstream</span><span class="w"> </span><span class="nt">homu</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="err">server</span><span class="w"> </span><span class="err">127.0.0.1:54856</span><span class="p">;</span>
<span class="p">}</span>
<span class="nt">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="err">listen</span><span class="w"> </span><span class="err">public.ip.</span><span class="n">address</span><span class="p">:</span><span class="mi">54856</span><span class="p">;</span>
<span class="w"> </span><span class="err">location</span><span class="w"> </span><span class="err">/github</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="err">proxy_set_header</span><span class="w"> </span><span class="err">Host</span><span class="w"> </span><span class="err">$http_host</span><span class="p">;</span>
<span class="w"> </span><span class="err">proxy_redirect</span><span class="w"> </span><span class="err">off</span><span class="p">;</span>
<span class="w"> </span><span class="err">proxy_set_header</span><span class="w"> </span><span class="err">X-Real-IP</span><span class="w"> </span><span class="err">$remote_addr</span><span class="p">;</span>
<span class="w"> </span><span class="err">proxy_set_header</span><span class="w"> </span><span class="err">X-Scheme</span><span class="w"> </span><span class="err">$scheme</span><span class="p">;</span>
<span class="w"> </span><span class="err">proxy_pass</span><span class="w"> </span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">homu</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="nt">location</span><span class="w"> </span><span class="o">~*</span><span class="w"> </span><span class="o">(?:^|/)</span><span class="err">\</span><span class="o">.</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="err">deny</span><span class="w"> </span><span class="err">all</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="err">}</span>
</code></pre></div>
<p>这样就可以隐藏 Homu 暴露的信息, 其他完整功能可以通过内网访问.</p>
<h2 id="2-buildbot">2. 搭建 buildbot</h2>
<p>buildbot 其官方文档也比较详细, 可以参考其官方文档进行搭建, 需要注意的是目前
<a href="https://github.com/barosl/homu">homu</a> 适配的 buildbot 是 0.8, 目前 buildbot-0.9 正在发布 beta, 和 0.8
有很多地方都不兼容.</p>
<p>这里着重说一下结合 pylint 做代码检查.</p>
<h3 id="_3">代码检查</h3>
<p>我们希望通过 buildbot 使用 pylint 来做代码检查, 但是考虑到历史遗留问题我们仅
希望对当前提交的变更做检查. 综合一下几个方面 <a href="https://github.com/sebdah/git-pylint-commit-hook">git-pylint-commit-hook</a> 可
以完美的满足需求:</p>
<ul>
<li><a href="https://github.com/sebdah/git-pylint-commit-hook">git-pylint-commit-hook</a> 配置成 pre-commit 可以对<code>将要被提交(git add 过)的文件</code>进行检查</li>
<li>通过 <code>git reset HEAD^ --soft</code> 可以将上一次提交的文件变成<code>将要被提交的文件</code></li>
<li><a href="https://github.com/barosl/homu">Homu</a> 在接受(r+)某一 PR 时会将变更通过 <code>--no-ff</code>(no fast-forward) 的方式合并到 <code>auto</code> 分支</li>
<li>通过 no fast-forward 合并相当于将所涉及的提交打了一个节点, 在通过 <code>git reset HEAD^ --soft</code> 进行重置时会将所涉及的提交的文件都变成<code>将要被提交的文件</code></li>
</ul>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>git<span class="w"> </span>checkout<span class="w"> </span>-b<span class="w"> </span>feature-test
$<span class="w"> </span><span class="c1"># edit file a</span>
$<span class="w"> </span>git<span class="w"> </span>add<span class="w"> </span>a<span class="w"> </span><span class="o">&&</span><span class="w"> </span>git<span class="w"> </span>commit<span class="w"> </span>-m<span class="w"> </span><span class="s1">'Change file a'</span>
$<span class="w"> </span><span class="c1"># edit file b</span>
$<span class="w"> </span>git<span class="w"> </span>add<span class="w"> </span>b<span class="w"> </span><span class="o">&&</span><span class="w"> </span>git<span class="w"> </span>commit<span class="w"> </span>-m<span class="w"> </span><span class="s1">'Change file b'</span>
$<span class="w"> </span>git<span class="w"> </span>checkout<span class="w"> </span>master
$<span class="w"> </span>git<span class="w"> </span>merge<span class="w"> </span>--no-ff<span class="w"> </span>feature-test
$<span class="w"> </span>git<span class="w"> </span>reset<span class="w"> </span>HEAD^<span class="w"> </span>--soft
$<span class="w"> </span>git<span class="w"> </span>status
On<span class="w"> </span>branch<span class="w"> </span>master
Changes<span class="w"> </span>to<span class="w"> </span>be<span class="w"> </span>committed:
<span class="w"> </span><span class="o">(</span>use<span class="w"> </span><span class="s2">"git reset HEAD <file>..."</span><span class="w"> </span>to<span class="w"> </span>unstage<span class="o">)</span>
<span class="w"> </span>modified:<span class="w"> </span>a
<span class="w"> </span>modified:<span class="w"> </span>b
</code></pre></div>
<p>buildbot 配置文件片段如下:</p>
<div class="highlight"><pre><span></span><code><span class="n">lint_factory</span> <span class="o">=</span> <span class="n">util</span><span class="o">.</span><span class="n">BuildFactory</span><span class="p">()</span>
<span class="n">lint_factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">steps</span><span class="o">.</span><span class="n">Git</span><span class="p">(</span><span class="n">repourl</span><span class="o">=</span><span class="n">repo</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s1">'full'</span><span class="p">,</span> <span class="n">branch</span><span class="o">=</span><span class="s1">'auto'</span><span class="p">,</span> <span class="n">progress</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
<span class="n">lint_factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">steps</span><span class="o">.</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s2">"pip"</span><span class="p">,</span> <span class="s2">"install"</span><span class="p">,</span> <span class="s2">"-U"</span><span class="p">,</span> <span class="s2">"-r"</span><span class="p">,</span> <span class="s2">"requirements.txt"</span><span class="p">]))</span>
<span class="n">lint_factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">steps</span><span class="o">.</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s2">"git"</span><span class="p">,</span> <span class="s2">"reset"</span><span class="p">,</span> <span class="s2">"HEAD^"</span><span class="p">,</span> <span class="s2">"--soft"</span><span class="p">]))</span>
<span class="n">lint_factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">steps</span><span class="o">.</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s2">"git-pylint-commit-hook"</span><span class="p">,</span> <span class="s2">"--limit=8"</span><span class="p">,</span> <span class="s2">"--pylintrc=./.pylintrc"</span><span class="p">]))</span>
<span class="n">c</span><span class="p">[</span><span class="s1">'builders'</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
<span class="n">util</span><span class="o">.</span><span class="n">BuilderConfig</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"auto-lint"</span><span class="p">,</span>
<span class="n">slavenames</span><span class="o">=</span><span class="p">[</span><span class="s2">"example-slave"</span><span class="p">],</span>
<span class="n">factory</span><span class="o">=</span><span class="n">lint_factory</span><span class="p">))</span>
</code></pre></div>
<p>可根据自身需求做变更, 比如 <code>--limit=8</code> 来限制最低分数, 低于这个分数将导致构建
失败.</p>
<p>buildbot 可自定义性很强, homu 是可以支持多个 builder 的. 所以除了代码检查我们
还有测试. Homu 在检测这两个 Builder 都构建成功后才会合并当前 PR.</p>
<h2 id="3">3. 工作流</h2>
<p>实现了如上功能之后我们对工作流进行了调整:</p>
<ol>
<li>通过主题分支开发并往长期分支提交 PR</li>
<li>reviewer 在 <code>LGTM</code> 后提交评论 <code>r+</code> 并 @ Homu 账号(@homu r+)</li>
<li>之后 Homu 会合并 PR 变更到 <code>auto</code> 分支</li>
<li>buildbot 监听到 <code>auto</code> 分支的变更后就会运行相应构建并在完成后后通知 Homu</li>
<li>Homu 在收到成功的通知后合并 PR 到长期分支</li>
</ol>
<h2 id="4">4. 已知问题</h2>
<p>Homu 在将 <code>auto</code> 分支合并到长期分支时会采用 <code>fast-forward</code> 的方式进行合并,
所以不会产生新的 commit, buildbot 0.8 对同一提交(sha 相同) 仅触发一次 build.
所以会导致其他一些比如自动部署的 builder 无法触发.</p>Python mock 使用心得2016-04-03T11:42:00+08:002016-04-03T11:42:00+08:00coldtag:www.linuxzen.com,2016-04-03:/python-mock-shi-yong-xin-de.html<p>好久没有更新博客, 趁着清明节小长假和我儿子正在睡觉更新一篇刷刷存在感.
近来变化很多, 儿子也有了, 工作上也有很 …</p><p>好久没有更新博客, 趁着清明节小长假和我儿子正在睡觉更新一篇刷刷存在感.
近来变化很多, 儿子也有了, 工作上也有很多收获. 这篇博客就分享一下关于 <code>mock</code> 的使用的心得体会.</p>
<p>很长一段时间以来写单元测试都类似写执行脚本, 运行一下然后看一下结果.
这里面有一部分原因是因为无法规避外部的依赖组件, 比如:</p>
<ul>
<li>数据库操作</li>
<li>外部接口调用</li>
<li>外部其他不可控因素</li>
</ul>
<p>这样写测试只关心当前测试的结果, 而不去管其他测试是否 <code>passed</code>.</p>
<p>后面随着团队开始进新人, 由于团队里每个人的标准和水平不同,
开始不得不重视整体项目的质量, 发现没有好的测试就没有统一的标准来衡量提交代码的质量,
当然说到代码质量还有另外一个和测试放在一起的标准就是代码风格, 这不是本文的主题所里这里就暂且不提.</p>
<p>为了能写好测试就不得不面对现实项目的复杂性, 诸如外部接口数据库操作等.
这时开始将目光转向 <code>mock</code>, 因为之前有听过类似概念, 但是还是有误解,
以为把要测的东西都模拟掉了还测试什么呢? 但是真正的了解 <code>mock</code> 之后才完整的理解了单元测试.</p>
<p><strong>单元测试应该只针对当前单元进行测试, 所有的外部依赖应该是稳定的, 在别处进行测试过的.</strong>
使用 <code>mock</code> 就可以对外部依赖组件实现进行模拟并且替换掉, 从而隐藏外部组件的实现,
使得单元测试将焦点只放在当前的逻辑(当前单元),</p>
<h2 id="_1">安装</h2>
<p><code>mock</code> 在 Python3 中是内置的, 直接 <code>import unittest.mock</code> 即可,
但是在 Python2 中是需要额外安装的, 安装完 <code>import mock</code> 即可</p>
<div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>mock
</code></pre></div>
<h2 id="_2">技巧</h2>
<p>在本文不详细介绍如何使用, 具体请参见<a href="https://docs.python.org/3/library/unittest.mock.html">官方文档</a>.
这里分享几个技巧.</p>
<p>安装官方文档给的示例一开始像下面这么使用 mock</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">DemoTestCase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="nd">@mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"pkg.mod.dep_mod.func"</span><span class="p">,</span> <span class="n">autospec</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_demo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func_mock</span><span class="p">):</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">return_value</span> <span class="o">=</span> <span class="kc">False</span>
<span class="c1"># real test code</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">)</span>
</code></pre></div>
<p>但是当整个测试都依赖这个组件时上面的使用方式就会产生大量的相同的初始化代码,
所以定义了一个装饰器向下面这样</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">unittest</span>
<span class="kn">import</span> <span class="nn">functools</span>
<span class="kn">import</span> <span class="nn">mock</span>
<span class="k">def</span> <span class="nf">_mock_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="nd">@functools</span><span class="o">.</span><span class="n">wraps</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<span class="nd">@mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"pkg.mod.dep_mod.func"</span><span class="p">,</span> <span class="n">autospec</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func_mock</span><span class="p">)</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">return_value</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func_mock</span><span class="p">)</span>
<span class="k">return</span> <span class="n">wrapper</span>
<span class="k">class</span> <span class="nc">DemoTestCase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="nd">@_mock_wrapper</span>
<span class="k">def</span> <span class="nf">test_demo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func_mock</span><span class="p">):</span>
<span class="c1"># real test code</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">)</span>
</code></pre></div>
<p>上面操作是很方便, 但是有几个缺点:</p>
<ul>
<li>代码不够清晰</li>
<li>如果增加 mock 组件则需要修改每一个被装饰函数接收的参数</li>
</ul>
<p>所以在仔细阅读官方文档后发现, <code>mock.patch</code> 返回一个对象, 可以通过 <code>start</code>/<code>stop</code> 方法来应用.
所以产生了下面的代码.</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">DemoTestCase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">DemoTestCase</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">setUp</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_patcher</span> <span class="o">=</span> <span class="n">mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"pkg.mod.dep_mod.func"</span><span class="p">,</span> <span class="n">autospec</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_func_mock</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_patcher</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_func_mock</span><span class="o">.</span><span class="n">return_value</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">tearDown</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">DemoTestCase</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">tearDown</span><span class="p">()</span>
<span class="c1"># cleanup</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_patcher</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_demo</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># real test code</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_func_mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">)</span>
</code></pre></div>
<p>当然如果有多个依赖组件需要 <code>mock</code> 可以将 <code>patcher</code> 存在一个列表里在 <code>tearDown</code> 方法里统一清理.</p>
<p><strong>不要忘记清理, 因为当前 <code>mock</code> 的组件应当仅在当前测试里生效, 如果忘记了清理可能会影响到其他测试.</strong></p>
<p>如果当前单元测试仅有部分测试依赖该组件也可以通过上下文管理的方式进行管理, 更加灵活.</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">contextlib</span>
<span class="k">class</span> <span class="nc">DemoTestCase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="nd">@contextlib</span><span class="o">.</span><span class="n">contextmanger</span>
<span class="k">def</span> <span class="nf">_mock_context</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">patcher</span> <span class="o">=</span> <span class="n">mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"pkg.mod.dep_mod.func"</span><span class="p">,</span> <span class="n">autospec</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">func_mock</span> <span class="o">=</span> <span class="n">patcher</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">return_value</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">yield</span> <span class="n">func_mock</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">patcher</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_demo</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">with</span> <span class="bp">self</span><span class="o">.</span><span class="n">_mock_context</span><span class="p">()</span> <span class="k">as</span> <span class="n">func_mock</span><span class="p">:</span>
<span class="c1"># real test code</span>
<span class="n">func_mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">)</span>
</code></pre></div>
<h2 id="_3">写在后面的话</h2>
<p>测试对一个项目很重要, 特别是对上了规模的项目, 上了 <code>mock</code> 之后我花了很长时间将已有项目的测试整理完毕,
使测试有效并且可以全部 <code>passed</code>.
有了良好的基础之后接下来就很方便的使用了一些集成工具来限制不规范的代码提交到长期分支,
这部分内容将放在下一篇去分享.</p>Python 内存泄露实战分析2015-03-30T17:29:00+08:002015-03-30T17:29:00+08:00coldtag:www.linuxzen.com,2015-03-30:/python-nei-cun-xie-lu-shi-zhan-fen-xi.html<h2 id="_1">引子</h2>
<p>之前一直盲目的认为 Python 不会存在内存泄露, 但是眼看着上线的项目随着运行时间的增长
而越来越大的内存占用, 我 …</p><h2 id="_1">引子</h2>
<p>之前一直盲目的认为 Python 不会存在内存泄露, 但是眼看着上线的项目随着运行时间的增长
而越来越大的内存占用, 我意识到我写的程序在发生内存泄露, 之前 debug 过
<a href="http://www.linuxzen.com/logging-mo-kuai-wu-yong-dao-zhi-de-nei-cun-xie-lu.html">logging 模块导致的内存泄露</a>.</p>
<p>目前看来, 还有别的地方引起的内存泄露. 经过一天的奋战, 终于找到了内存泄露的地方, 目前项目
跑了很长时间, 在业务量较小的时候内存还是能回到刚启动的时候的内存占用.</p>
<h2 id="_2">什么情况下不用这么麻烦</h2>
<p>如果你的程序只是跑一下就退出大可不必大费周章的去查找是否有内存泄露, 因为 Python 在退出时
会释放它所分配的所有内存, 如果你的程序需要连续跑很长时间那么就要仔细的查找是否
产生了内存泄露.</p>
<h2 id="_3">场景</h2>
<p>如何产生的内存泄露呢, 项目是一个 TCP server, 每当有连接过来时都会创建一个连接实例来进行
管理, 每次断开时连接实例还被占用并没有释放. 没有被释放的原因肯定是因为有某个地方对连接
实例的引用没有释放, 所以随着时间的推移, 连接创建分配内存, 连接断开并没有释放掉内存, 所以
就会产生内存泄露.</p>
<h2 id="_4">调试方法</h2>
<p>由于不知道具体是哪里引起的内存泄露, 所以要耐心的一点点调试. </p>
<p>由于知道了断开连接时没有释放, 所以我就不停的模拟创建连接然后发送一些包后断开连接,
然后通过下面一行 shell 来观察内存占用情况:</p>
<div class="highlight"><pre><span></span><code><span class="nv">PID</span><span class="o">=</span><span class="m">50662</span><span class="p">;</span><span class="k">while</span><span class="w"> </span>true<span class="p">;</span><span class="w"> </span><span class="k">do</span><span class="p">;</span><span class="w"> </span>ps<span class="w"> </span>aux<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="nv">$PID</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span>grep<span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span><span class="s1">'{print $5" "$6}'</span><span class="w"> </span>>><span class="w"> </span>t<span class="p">;</span><span class="w"> </span>sleep<span class="w"> </span><span class="m">1</span><span class="p">;</span><span class="w"> </span><span class="k">done</span>
</code></pre></div>
<p>如果在增长了一定的量后保持住就说明已经没有产生泄露.</p>
<p>同时可以在对象该释放的时候查看对象的引用计数, 通过 <code>sys.getrefcount(obj)</code>. 如果引用计数变为了 <code>2</code>
则说明该对象在跳出命名空间后就会被正确回收.</p>
<h2 id="_5">产生原因</h2>
<p>项目中两种情况导致对象没有被正确回收:</p>
<ul>
<li>被退出才回收的对象引用</li>
<li>交叉引用</li>
</ul>
<h3 id="_6">被退出才回收的对象引用</h3>
<p>为了追踪连接所以把连接对象同时放在一个列表里, 而这个列表只有在程序退出时才会被回收,
如果不正确处理, 那么分配的对象将也会只在程序退出时才会被回收.</p>
<p>全局变量和类变量都只会在程序退出的时候才会被回收:</p>
<div class="highlight"><pre><span></span><code><span class="n">_CONNECTIONS</span> <span class="o">=</span> <span class="p">[]</span>
<span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">Connection</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">server_loop</span><span class="p">():</span>
<span class="c1"># ...</span>
<span class="n">sock</span><span class="p">,</span> <span class="n">address</span> <span class="o">=</span> <span class="n">server_sock</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span>
<span class="n">connection</span> <span class="o">=</span> <span class="n">Connection</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="n">_CONNECTIONS</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">connection</span><span class="p">)</span>
<span class="c1"># ...</span>
<span class="n">sock</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</code></pre></div>
<p>上面把所有建立的连接都放在全局变量 <code>_CONNECTIONS</code> 里, 如果在关闭的时候不从这个列表
里取出(减少引用)则 connection 对象就不会被回收, 则每建立一次连接就会有个连接对象和连接
对象引用的对象不会被回收.</p>
<p>如果把对象放在一个类属性里也是一样的, 因为类对象在程序一开始就分配, 并在程序退出时才被回收.</p>
<p>解决办法就是在退出时从列表(或其他对象)里解除对对象的引用(删除)</p>
<div class="highlight"><pre><span></span><code><span class="n">_CONNECTIONS</span> <span class="o">=</span> <span class="p">[]</span>
<span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">Connection</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">server_loop</span><span class="p">():</span>
<span class="c1"># ...</span>
<span class="n">sock</span><span class="p">,</span> <span class="n">address</span> <span class="o">=</span> <span class="n">server_sock</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span>
<span class="n">connection</span> <span class="o">=</span> <span class="n">Connection</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="n">_CONNECTIONS</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">connection</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># ...</span>
<span class="n">sock</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">_CONNECTIONS</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">connection</span><span class="p">)</span> <span class="c1"># XXX</span>
</code></pre></div>
<h3 id="_7">交叉引用</h3>
<p>有时候我们为对象分配一个实例属性时需要将自己本身赋值给实例属性, 作为实例属性的实例属性,
说着很拗口, 看一下代码:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">ConnectionHandler</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">connection</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_conn</span> <span class="o">=</span> <span class="n">connection</span>
<span class="k">class</span> <span class="nc">Connection</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_conn_handler</span> <span class="o">=</span> <span class="n">ConnectionHandler</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="c1"># XXX</span>
</code></pre></div>
<p>上面的代码就会产生交叉引用, 交叉引用会让解释器困惑, 从而之后只能靠2代和3代回收, 这个过程可能会很慢.</p>
<p>解决这种问题的方法就是使用 <code>弱引用</code></p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">weakref</span>
<span class="k">class</span> <span class="nc">ConnectionHandler</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">connection</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_conn</span> <span class="o">=</span> <span class="n">connection</span>
<span class="k">class</span> <span class="nc">Connection</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sock</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_conn_handler</span> <span class="o">=</span> <span class="n">ConnectionHandler</span><span class="p">(</span><span class="n">weakref</span><span class="o">.</span><span class="n">proxy</span><span class="p">(</span><span class="bp">self</span><span class="p">))</span> <span class="c1"># XXX</span>
</code></pre></div>
<h3 id="_8">日志模块</h3>
<p>参见 <a href="http://www.linuxzen.com/logging-mo-kuai-wu-yong-dao-zhi-de-nei-cun-xie-lu.html">logging 模块导致的内存泄露</a>.</p>
<h2 id="_9">参考</h2>
<p><a href="http://www.cnblogs.com/vamei/p/3232088.html">Python 垃圾回收</a></p>logging 模块误用导致的内存泄露2015-01-31T10:58:00+08:002015-01-31T10:58:00+08:00coldtag:www.linuxzen.com,2015-01-31:/logging-mo-kuai-wu-yong-dao-zhi-de-nei-cun-xie-lu.html<p>首先介绍下怎么发现的吧, 线上的项目日志是通过 <code>logging</code> 模块打到 syslog 里,
跑了一段时间后发现 syslog 的 UDP 连接超过了 8W, 没错是 8 …</p><p>首先介绍下怎么发现的吧, 线上的项目日志是通过 <code>logging</code> 模块打到 syslog 里,
跑了一段时间后发现 syslog 的 UDP 连接超过了 8W, 没错是 8 W. 主要是 logging
模块用的不对</p>
<p>我们之前有这么一个需求, 就是针对每一个连接日志输出当前连接的信息, 所以每一个
连接就创建了一个日志实例, 并分配一个 <code>Formatter</code>, 创建日志实例为了区分其他连接
所以我就简单粗暴的用了当前对象的 id 来作为日志名称:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">logging</span>
<span class="k">class</span> <span class="nc">Connection</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_logger_name</span> <span class="o">=</span> <span class="s2">"Connection.</span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">id</span><span class="p">(</span><span class="bp">self</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_logger_name</span><span class="p">)</span>
</code></pre></div>
<p>当然测试环境是开 DEBUG, 开 DEBUG 就不会往 syslog 里打, 所以不会出现 UDP 连接数
过多, 也就不会知道有内存泄露的, 我们来看看这样为什么会导致内存泄露, 首先看看
<code>getLogger</code> 的代码:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">getLogger</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> Return a logger with the specified name, creating it if necessary.</span>
<span class="sd"> If no name is specified, return the root logger.</span>
<span class="sd"> """</span>
<span class="k">if</span> <span class="n">name</span><span class="p">:</span>
<span class="k">return</span> <span class="n">Logger</span><span class="o">.</span><span class="n">manager</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">root</span>
</code></pre></div>
<p>主要调用了 <code>Logger.manager.getLogger</code>, 这个函数有下面一段代码片段</p>
<div class="highlight"><pre><span></span><code> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">loggerDict</span><span class="p">:</span>
<span class="n">rv</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">loggerDict</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">rv</span><span class="p">,</span> <span class="n">PlaceHolder</span><span class="p">):</span>
<span class="n">ph</span> <span class="o">=</span> <span class="n">rv</span>
<span class="n">rv</span> <span class="o">=</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">loggerClass</span> <span class="ow">or</span> <span class="n">_loggerClass</span><span class="p">)(</span><span class="n">name</span><span class="p">)</span>
<span class="n">rv</span><span class="o">.</span><span class="n">manager</span> <span class="o">=</span> <span class="bp">self</span>
<span class="bp">self</span><span class="o">.</span><span class="n">loggerDict</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">rv</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fixupChildren</span><span class="p">(</span><span class="n">ph</span><span class="p">,</span> <span class="n">rv</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fixupParents</span><span class="p">(</span><span class="n">rv</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">rv</span> <span class="o">=</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">loggerClass</span> <span class="ow">or</span> <span class="n">_loggerClass</span><span class="p">)(</span><span class="n">name</span><span class="p">)</span>
<span class="n">rv</span><span class="o">.</span><span class="n">manager</span> <span class="o">=</span> <span class="bp">self</span>
<span class="bp">self</span><span class="o">.</span><span class="n">loggerDict</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">rv</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fixupParents</span><span class="p">(</span><span class="n">rv</span><span class="p">)</span>
</code></pre></div>
<p>logging 模块为了保证同一个名称引用同一个日志实例,所以就把所有的日志实例全部存
在了一个 <code>loggerDict</code> 的字典里, 所以除非程序退出, 创建的日志实例引用是不会释放的,
所以日志实例里的 <code>handlers</code> 也不会释放. 之前我又用的对象的 id 来作为日志名称
的一部分, 所以 <code>SyslogHandler</code> 创建的 UDP 连接就一直被占用导致了过多的 UDP 连接. </p>
<p>为了解决这个问题我在连接关闭的时候加入了如下代码:</p>
<div class="highlight"><pre><span></span><code><span class="n">logging</span><span class="o">.</span><span class="n">Logger</span><span class="o">.</span><span class="n">manager</span><span class="o">.</span><span class="n">loggerDict</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_logger_name</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">manager</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">handlers</span> <span class="o">=</span> <span class="p">[]</span>
</code></pre></div>
<p>按说只加上上面第一行的代码就应该释放了, 但是没有, 所以又有了第三行代码, SyslogHandler
才最终释放, 这个问题暂时还不知道为什么, 还需要再查查.</p>
<p><strong>2015-03-30 更新</strong>
如果日志名称是以 <code>.</code> 分隔, logging 模块则会将最后一部分作为日志名, 并往上去寻找 <code>父 Logger</code>,
如果找不到则创建 <code>PlaceHolder</code> 对象作为父, 并引用 <code>Logger</code>.</p>
<p>比如创建的 Logger 名称为 <code>a.b.c</code>, 那么实际的名称则为 <code>c</code>, 并将 <code>b</code> 作为 <code>c</code> 的父, <code>a</code> 作为 <code>b</code> 的
父, 如果没有该名称的 <code>Logger</code> 则创建 <code>PlaceHolder</code> 对象作为代替, <code>PlaceHolder</code> 会创建对当前 <code>Logger</code>
的引用. 所以需要被回收的日志对象名称里不应包含 <code>.</code></p>基于 Python 生成器的 Tornado 协程异步2014-12-19T17:15:00+08:002014-12-19T17:15:00+08:00coldtag:www.linuxzen.com,2014-12-19:/ji-yu-python-sheng-cheng-qi-de-tornado-xie-cheng-yi-bu.html<p><a href="http://www.tornadoweb.org/en/branch4.0/releases/v4.0.0.html">Tornado 4.0</a> 已经发布了很长一段时间了,
新版本广泛的应用了协程(<code>Future</code>)特性. 我们目前已经将 Tornado 升级到最新版本, 而且也大量的 …</p><p><a href="http://www.tornadoweb.org/en/branch4.0/releases/v4.0.0.html">Tornado 4.0</a> 已经发布了很长一段时间了,
新版本广泛的应用了协程(<code>Future</code>)特性. 我们目前已经将 Tornado 升级到最新版本, 而且也大量的使用协程特性. </p>
<p>很长时间没有更新博客, 今天就简单介绍下 Tornado 协程实现原理, Tornado 的协程是基于 Python 的生成器实现的,
所以首先来回顾下生成器.</p>
<h2 id="_1">生成器</h2>
<p>Python 的生成器可以保存执行状态 并在下次调用的时候恢复, 通过在函数体内使用 <code>yield</code> 关键字
来创建一个生成器, 通过内置函数 <code>next</code> 或生成器的 <code>next</code> 方法来恢复生成器的状态. </p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="k">yield</span> <span class="mi">1</span>
</code></pre></div>
<p>我们调用 <code>test</code> 函数, 此时并不会返回结果, 而是会返回一个生成器</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">test</span><span class="p">()</span>
<span class="o"><</span><span class="n">generator</span> <span class="nb">object</span> <span class="n">test</span> <span class="n">at</span> <span class="mh">0x100b3b320</span><span class="o">></span>
</code></pre></div>
<p>我们调用其 <code>next</code> 方法则返回 <code>yield</code> 关键字之后的内容.</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">t</span> <span class="o">=</span> <span class="n">test</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">t</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
<span class="mi">1</span>
</code></pre></div>
<p>如果我们接着调用 <code>next</code> 方法, 后面又没有 <code>yield</code> 关键字继续返回的话, 会抛出一个
<code>StopIteration</code> 异常.</p>
<p><code>yield</code> 关键字不仅仅能从生成器内部返回状态, 同时也可以将外部信息传递到生成器内部,
通过将 <code>yeild</code> 关键里赋值给变量, 并调用生成器的 <code>send</code> 方法来将对象传递到生成器
内部. 需要注意的是生成器的开始必须调用其 <code>next</code> 方法, 后面 <code>send</code> 方法调用的同时
也会触发 <code>next</code> 动作. 如果没有变量接收 <code>yield</code> 关键字那么 <code>send</code> 传递的值将会
被丢弃.</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">a</span> <span class="o">=</span> <span class="k">yield</span>
<span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</code></pre></div>
<p>首先调用 <code>next</code> 上面函数返回的生成器将返回 <code>None</code>, 如果这时候直接调用 <code>next</code> 将
会给生成器发送 <code>None</code>, 如果调用 <code>send</code> 发送一个值, 将打印这个值并抛出 <code>StopIteration</code>
异常.</p>
<h2 id="_2">一个简单地协程</h2>
<p>以上就是实现协程的所有基础, 为了加深理解, 我们这里写一个小例子, 例子我们只使用协程
开启两个甚至多个死循环, 下面就是一个极其简单地例子::</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">absolute_import</span><span class="p">,</span> <span class="n">print_function</span><span class="p">,</span> <span class="n">division</span><span class="p">,</span> <span class="n">with_statement</span>
<span class="k">def</span> <span class="nf">loop1</span><span class="p">():</span>
<span class="w"> </span><span class="sd">""" 循环1负责抛出一个函数和对应的参数, 并接收结果</span>
<span class="sd"> """</span>
<span class="n">a</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">ret</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">ret</span> <span class="o">=</span> <span class="k">yield</span> <span class="nb">sum</span><span class="p">,</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">ret</span><span class="p">]</span>
<span class="n">a</span><span class="p">,</span> <span class="n">ret</span> <span class="o">=</span> <span class="n">ret</span><span class="p">,</span> <span class="n">a</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Loop1 ret"</span><span class="p">,</span> <span class="n">ret</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">loop2</span><span class="p">():</span>
<span class="w"> </span><span class="sd">""" 循环2 负责接收函数并计算结果, 然后 yield 出结果</span>
<span class="sd"> """</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">func</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="k">yield</span>
<span class="k">yield</span> <span class="n">func</span><span class="p">(</span><span class="n">args</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Loop2"</span><span class="p">)</span>
<span class="n">l1</span> <span class="o">=</span> <span class="n">loop1</span><span class="p">()</span>
<span class="n">l2</span> <span class="o">=</span> <span class="n">loop2</span><span class="p">()</span>
<span class="n">tmp</span> <span class="o">=</span> <span class="n">l1</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">l2</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
<span class="n">ret</span> <span class="o">=</span> <span class="n">l2</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">tmp</span><span class="p">)</span>
<span class="n">tmp</span> <span class="o">=</span> <span class="n">l1</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">ret</span><span class="p">)</span>
</code></pre></div>
<p>上面例子里 loop1 负责产生任务, loop2 负责执行任务, 主循环负责调度任务并将任务结果发回给
任务产生者.</p>
<h2 id="tornado">Tornado 如何做的</h2>
<p>我们首先看一个使用 Tornado 协程异步的例子</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">absolute_import</span><span class="p">,</span> <span class="n">print_function</span><span class="p">,</span> <span class="n">division</span><span class="p">,</span> <span class="n">with_statement</span>
<span class="kn">from</span> <span class="nn">tornado</span> <span class="kn">import</span> <span class="n">gen</span>
<span class="kn">from</span> <span class="nn">tornado</span> <span class="kn">import</span> <span class="n">web</span>
<span class="kn">from</span> <span class="nn">tornado</span> <span class="kn">import</span> <span class="n">httpclient</span>
<span class="k">class</span> <span class="nc">ActionHandler</span><span class="p">(</span><span class="n">web</span><span class="o">.</span><span class="n">RequestHandler</span><span class="p">):</span>
<span class="nd">@gen</span><span class="o">.</span><span class="n">coroutine</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="k">yield</span> <span class="n">httpclient</span><span class="o">.</span><span class="n">AsyncHTTPClient</span><span class="p">()</span><span class="o">.</span><span class="n">fetch</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">)</span>
<span class="c1"># ...</span>
</code></pre></div>
<p>其实原理在上面简单地例子里已经讲清楚了, 我们来简单分析一遍上面的例子, 首先 Tornado 得到
<code>ActionHandler.get</code> 方法抛出(<code>next</code>)的一个任务, 然后异步的去执行任务, 当任务(网络请求)结束或
异常时 Tornado 取得事件通知然后将结果放回(<code>send</code>)到该方法中让该方法继续执行.</p>
<p>由于是异步的, 调用这个方法并不会阻塞其他任务执行.</p>
<p>这时候我们的方法其实就是上个例子 <code>loop1</code> 函数, 而 <code>Tornado</code> 调度并执行了其抛出的任务.</p>
<h2 id="_3">总结</h2>
<p>Tornado 的协程异步可以让异步看起来是顺序执行的, 可以从一大串的 <code>callback</code> 中解脱出来.</p>
<p><code>Tornado</code> 的协程异步并不是这三言两语能说清楚的, 其中有很复杂的封装和传递, 有兴趣可以自己
阅读源码.</p>Python 入门指南2014-05-23T14:29:00+08:002014-05-23T14:29:00+08:00coldtag:www.linuxzen.com,2014-05-23:/python-ru-men-zhi-nan.html<h1 id="_1">引子</h1>
<p>经常能在 Python 群里看到很多新人在问一些非常基础的问题, 基本每天都在重复的问这些问题,
在这里就总结一下这些 …</p><h1 id="_1">引子</h1>
<p>经常能在 Python 群里看到很多新人在问一些非常基础的问题, 基本每天都在重复的问这些问题,
在这里就总结一下这些问题.</p>
<p>首先声明, 本文不打算教会你 Python, 本文力图陈列一些新手容易遇到的问题, 并企图教会你
如何学习 Python, 在遇到问题的时候如何提问.</p>
<h1 id="_2">关于版本</h1>
<p>学习 Python 的第一步需要选择版本, Python 3.x 和 2.x 的断层较大, 3.x 不向后兼容 2.x.
Python 现在主流应该还是 Python 2.7, Python 2.7 将会是 Python 2.x 的最后一个版本, 并且
会支持到 2020 年. 但是 Python 3 也在健康发展, 会慢慢取代 Python 2.7 成为主流版本.</p>
<h2 id="_3">选择版本</h2>
<p>在你要开始学习 Python 之前就是要确定要学习的版本,
不管你是选择 2 还是 3, 虽然有差别, 但不是很大, 等你熟悉了之后就可以触类旁通.
不必太害怕选择了一个版本到时候无法兼顾另一个版本.</p>
<p>如果你无法确定要学习的版本,可以根据以下方法来确定要学习的版本.</p>
<h3 id="_4">手上已有的教程</h3>
<p>如果你手上已经有了一本关于 Python 基础的书, 那么书的开头应该会交代 Python 版本.
那就根据这个教程选择要学习的版本.</p>
<h3 id="_5">要上手的项目</h3>
<p>如果你已经有一个 Python 项目等着你去上手, 那么先了解项目需要什么版本. 然后根据
需要的版本找支持对应版本的基础书籍.</p>
<h3 id="python-3">直接选择 Python 3</h3>
<p>如果你没有以上的负担, 那么推荐你直接学习 Python 3, 但是你要找到一本支持 Python 3
的入门书籍, 不然你前期你会发现所有的都是错的, 会直接打消你的自信心.</p>
<p>Python 3 的底层全部用 unicode 实现, 所以不会遇到 Python2 烦人的 <code>UnicodeDecodeError</code>
类似的异常, 关于这个后面会讲到.</p>
<h2 id="_6">版本差异</h2>
<p>Python 官网已经有详细的版本差异(<a href="https://docs.python.org/3/whatsnew/3.0.html">这里</a>)</p>
<p>这里简单列出几个主要差异</p>
<h3 id="print">print 的改变</h3>
<p>在 Python2 里 print 是一个语句, 用以下方式输出</p>
<div class="highlight"><pre><span></span><code><span class="nb">print</span> <span class="s1">'Hello world!'</span>
</code></pre></div>
<p>在 Python 3 里 print 变成了一个 函数</p>
<div class="highlight"><pre><span></span><code><span class="nb">print</span><span class="p">(</span><span class="s1">'Hello world'</span><span class="p">)</span>
</code></pre></div>
<h3 id="_7">输入函数的改变</h3>
<p>在 Python 2 里用 <code>raw_input</code> 函数获取输入</p>
<div class="highlight"><pre><span></span><code><span class="n">raw_input</span><span class="p">(</span><span class="s2">"Enter your name: "</span><span class="p">)</span>
</code></pre></div>
<p>在 Python 3 里用 <code>input</code> 函数代替 </p>
<div class="highlight"><pre><span></span><code><span class="nb">input</span><span class="p">(</span><span class="s2">"Enter your name: "</span><span class="p">)</span>
</code></pre></div>
<p>当然还有很多, 这里不一一列举, 如果你以后对 Python 有了一定了解, 可以看看
<a href="https://www.ibm.com/developerworks/cn/linux/l-python3-1/">这篇</a>文章</p>
<h2 id="_8">多版本共存</h2>
<p>Python 是可以多个版本共存的, 如果可以你可以同时安装 Python 2 和 3, 自己
动手亲自比较一下.</p>
<h2 id="_9">代码兼容</h2>
<p>Python 2 和 3 是可以通过一些技巧来实现兼容的, 这点超出了本文讨论的范畴,
如想了解可以上网搜索.</p>
<h1 id="python-shell">关于 Python Shell</h1>
<p>Python Shell 就是你在命令行下运行 <code>python</code> 指令后出来的一个交互式 shell,
或者运行 Windows 下的 <code>IDLE</code> 出来的窗口叫做 Python Shell, Python Shell
提供一种 "所见即所得" 的方式来运行 Python 语句, 这将是你学习 Python
的一种重要工具.</p>
<h2 id="python-shell_1">区分 Python Shell 和 命令</h2>
<p>但是我见到很多人问像下面那样运行脚本为什么会出错</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">python</span> <span class="n">script</span><span class="o">.</span><span class="n">py</span>
</code></pre></div>
<p>这样是错误的, Python Shell 是运行 Python 语句的, 而 <code>python script.py</code> 是
一条命令, 意为运行 <code>script.py</code> 这个文件里的 Python 语句.</p>
<p>我们真正要做的是在命令行下执行这个命令, 所谓命令行就是 Windows 下
Win+R 输入 cmd 回车弹出的窗口.</p>
<p>如果你在命令行下运行失败请上网搜索了解关于 <code>PATH</code> 的相关知识.</p>
<h1 id="python-2">关于 Python 2</h1>
<p>如果你选择了 Python 2, 那么就有需要面对一些问题.</p>
<h2 id="_10">中文字符</h2>
<p>如果你的 Python 源码文件里出现了中文字符, 你就会发现无法运行出现</p>
<div class="highlight"><pre><span></span><code><span class="n">SyntaxError</span><span class="o">:</span><span class="w"> </span><span class="n">Non</span><span class="o">-</span><span class="n">ASCII</span><span class="w"> </span><span class="n">character</span><span class="w"> </span><span class="s1">'\xe5'</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="n">xx</span><span class="o">.</span><span class="na">py</span><span class="w"> </span><span class="n">on</span><span class="w"> </span><span class="n">line</span><span class="w"> </span><span class="mi">8</span><span class="o">,</span><span class="w"> </span><span class="n">but</span><span class="w"> </span><span class="n">no</span><span class="w"> </span><span class="n">encoding</span><span class="w"> </span><span class="n">declared</span><span class="o">;</span><span class="w"> </span><span class="n">see</span><span class="w"> </span><span class="n">http</span><span class="o">://</span><span class="n">www</span><span class="o">.</span><span class="na">python</span><span class="o">.</span><span class="na">org</span><span class="sr">/peps/</span><span class="n">pep</span><span class="o">-</span><span class="mi">0263</span><span class="o">.</span><span class="na">html</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">details</span>
</code></pre></div>
<p>你只需要在文件的最上面加上一行</p>
<div class="highlight"><pre><span></span><code><span class="c1">#-*- coding: utf-8 -*-</span>
</code></pre></div>
<p>即可</p>
<h2 id="unicodedecodeerror">UnicodeDecodeError 异常</h2>
<p>如果你不幸遇到了这个错误, 那么一般是因为字符串连接引起的, 比如下面这样的代码</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="s1">'中国'</span> <span class="o">+</span> <span class="sa">u</span><span class="s1">'a'</span>
</code></pre></div>
<p>如果你学过了基础你就会知道 <code>u''</code> 包围的字符串是 <code>unicode</code>, Python 2 里有两种类型的
字符串 <code>str</code> 和 <code>unicode</code>, 上面的 <code>'中国'</code> 就是 <code>str</code> 类型, <code>u'a'</code> 就是 <code>unicode</code>
类型.</p>
<p>如果这两种类型的字符串相连, <code>str</code> 类型的字符串会向 <code>unicode</code> 做隐式转化, 而隐式转换
默认的编码是 <code>ascii</code>, 明显 <code>ascii</code> 编码不可能包含任何汉字, 所以就会抛出这个异常.</p>
<p>如果上面反过来就不会抛出异常, 因为 <code>ascii</code> 里包含 <code>a</code> 这个字符</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="sa">u</span><span class="s1">'中国'</span> <span class="o">+</span> <span class="s1">'a'</span>
</code></pre></div>
<p>在文件中使用统一的类型的字符串可以规避这个问题, 要么都使用 <code>u''</code> 包围的字符串,
要么都使用 <code>''</code> 包围的字符串</p>
<p>当然上面单引号是可以换成双引号的.</p>
<h1 id="traceback">关于 Traceback</h1>
<p>Python 在程序出错的时候会向终端打印一串略长的信息叫做 <code>Traceback</code>, 像下面这样:</p>
<div class="highlight"><pre><span></span><code><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
<span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">16</span><span class="p">,</span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
<span class="n">main</span><span class="p">()</span>
<span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">14</span><span class="p">,</span> <span class="ow">in</span> <span class="n">main</span>
<span class="n">test</span><span class="p">()</span>
<span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">10</span><span class="p">,</span> <span class="ow">in</span> <span class="n">test</span>
<span class="s1">'中国'</span> <span class="o">+</span> <span class="sa">u</span><span class="s1">'a'</span>
<span class="ne">UnicodeDecodeError</span><span class="p">:</span> <span class="s1">'ascii'</span> <span class="n">codec</span> <span class="n">can</span><span class="s1">'t decode byte 0xe4 in position 0: ordinal not in range(128)</span>
</code></pre></div>
<p>这一段信息很详细的描述了出错的地方和详细的调用信息, 当然还有错误描述.</p>
<p>看懂 Traceback 将会有助于你更好的学习 Python. 这段 Traceback 说明</p>
<p>在 test.py 的第 16 行 main 函数里, 调用了在 14 行的 test 函数,
test 函数里文件的第 10 行的语句触发了 UnicodeDecodeError 异常.</p>
<p>简直太清晰了, 如果你觉得不清晰的话就怪我描述的不好吧.</p>
<h1 id="_11">关于第三方库</h1>
<p>Python 有大量的第三方库, 并且有 <code>setuptools</code> 工具可以安装这些库, setuptools
提供了 <code>easy_install</code> 命令可以从网上自动下载并安装第三方库, 可以参见
<a href="https://pypi.python.org/pypi/setuptools#installation-instructions">这里</a></p>
<h1 id="_12">关于提问</h1>
<p>如果你遇到了问题, 解决不了需要提问的时候, 请尽量的提供你的代码和详细的 Traceback.
代码直接发出来不是很好的方式, 请尽量贴到支持代码高亮的网站上, 并保持缩进.</p>
<p>推荐的下面两个贴代码的网站:</p>
<ul>
<li><a href="http://pastebin.com">http://pastebin.com</a></li>
<li><a href="http://paste.ubuntu.org.cn">http://paste.ubuntu.org.cn</a></li>
</ul>
<h1 id="_13">关于工具</h1>
<h2 id="_14">编辑器</h2>
<p>如果在学习初期并不推荐 IDE 作为开发工具, 使用文本编辑器可能有助于你的学习,
按照困难程度由低到高推荐下面几种文本编辑器:</p>
<ul>
<li><a href="https://code.google.com/p/ulipad/">ulipad</a></li>
<li><a href="http://notepad-plus-plus.org/">notepad++</a></li>
<li><a href="http://www.sublimetext.com/">Sublime Text</a></li>
<li><a href="http://vim.org">Vim</a> / <a href="http://www.gnu.org/software/emacs/">Emacs</a></li>
</ul>
<p>如果你想要一款功能强大的 IDE 那么推荐你 <a href="http://www.jetbrains.com/pycharm/">PyCharm</a></p>
<h2 id="_15">其他工具</h2>
<ul>
<li><a href="http://ipython.org">IPython</a> 一个 Python Shell 增强工具</li>
<li><a href="http://virtualenv.readthedocs.org/en/latest/">virtualenv</a> 可以用它获取一个干净的 Python 开发环境</li>
</ul>
<h1 id="_16">关于书籍</h1>
<p>Python 基础书籍有很多: 《简明 Python 教程》, 《Python 核心编程》《Python 学习手册》《Python参考手册》
等都是很不错的入门书籍.</p>
<p>如果你已经掌握了 Python 基础, 想继续阅读深入 Python, 这里推荐两本 Python 进阶书籍:
《Python 高级编程》《Python 标准库》</p>
<h1 id="_17">关于代码规范</h1>
<p>良好的代码规范可以让你的程序更加的简洁、美观和易读. Python 有自己的代码
规范, 可以参见 <a href="http://legacy.python.org/dev/peps/pep-0008/">PEP8</a></p>
<h1 id="_18">写在后面的话</h1>
<p>零零散散的介绍了一些, 在此抛砖引玉欢迎大家补充. 编写 Python 是快乐的,
希望本文可以对开始学 Python 的朋友有点帮助, 祝你们学习愉快.</p>PyQt 中用 QtNetwork 异步发起HTTP请求2014-05-07T15:52:00+08:002014-05-07T15:52:00+08:00coldtag:www.linuxzen.com,2014-05-07:/pyqt-zhong-yong-qtnetwork-yi-bu-fa-qi-httpqing-qiu.html<h2 id="_1">引子</h2>
<p>最近有需求要在 PyQt 中请求一个链接, 因为比较简单直接用 urllib2 处理了, 但是 urllib2 在
有延时的时候会造成 GUI 界面卡死. 所以 …</p><h2 id="_1">引子</h2>
<p>最近有需求要在 PyQt 中请求一个链接, 因为比较简单直接用 urllib2 处理了, 但是 urllib2 在
有延时的时候会造成 GUI 界面卡死. 所以今天研究研究 QtNetwork 模块.</p>
<p>QtNetwork 中的请求在 PyQt 中都是异步的.</p>
<h2 id="qhttp">简单的请求 QHttp</h2>
<h3 id="get">发起一个GET请求</h3>
<p><code>PyQt4.QtNetwork.QHttp</code> 可以发起一个简单请求, 需要注意的是这个对象需要通过调用
<code>setHost</code> 设置请求主机, 然后 调用 <code>get</code>/<code>post</code> 传入 <code>path</code> 才能正常使用.</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span><span class="p">,</span> <span class="n">QtNetwork</span>
<span class="k">class</span> <span class="nc">MainWidget</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MainWidget</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QHttp</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="c1"># 绑定 done 信号</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">done</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">on_req_done</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"http://linuxzen.com/"</span><span class="p">)</span>
<span class="c1"># 设置主机</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">setHost</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">host</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">port</span><span class="p">(</span><span class="mi">80</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">getId</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">path</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">on_req_done</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">error</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Success"</span>
<span class="nb">print</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">readAll</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Error"</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">main</span> <span class="o">=</span> <span class="n">MainWidget</span><span class="p">()</span>
<span class="n">main</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</code></pre></div>
<h3 id="_2">保存文件</h3>
<p>如果想要下载文件, 可以给 <code>get</code> 方法传一个 <code>QtCore.QFile</code> 对象, 会将请求内容保存
到文件</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span><span class="p">,</span> <span class="n">QtNetwork</span>
<span class="k">class</span> <span class="nc">MainWidget</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MainWidget</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QHttp</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">done</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">on_req_done</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com/"</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">setHost</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">host</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">port</span><span class="p">(</span><span class="mi">80</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">out</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QFile</span><span class="p">(</span><span class="s2">"./test"</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">getId</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">path</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">out</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">on_req_done</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
<span class="c1"># print self.http.readAll(), error</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">error</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Success"</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Error"</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">main</span> <span class="o">=</span> <span class="n">MainWidget</span><span class="p">()</span>
<span class="n">main</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</code></pre></div>
<p>上面代码将请求下来的内容保存到当前目录的 test 文件里.</p>
<h3 id="_3">处理头部信息</h3>
<p>现在有一个需求就是有一个链接会返回一个 301 或 302 的跳转, 但是 <code>PyQt4.QtNetwork.QHttp</code> 没有实现自动跳转, 需要我们自动判断头信息进行跳转.</p>
<p>我们可以绑定<code>PyQt4.QtNetwork.QHttp.responseHeaderReceived</code>的信号来处理头部信息,
这个信号将给槽传递一个 <code>PyQt4.QtNetwork.QHttpResponseHeader</code> 的实例.</p>
<p>通过判断状态吗, 并抓取 <code>Location</code> 头实现跳转</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span><span class="p">,</span> <span class="n">QtNetwork</span>
<span class="k">class</span> <span class="nc">MainWidget</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MainWidget</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QHttp</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">done</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">on_req_done</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">responseHeaderReceived</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">on_response_header</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"http://t.cn/zTocACq"</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">setHost</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">host</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">port</span><span class="p">(</span><span class="mi">80</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">out</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QFile</span><span class="p">(</span><span class="s2">"./test"</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">getId</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">path</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">out</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">on_response_header</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">response_header</span><span class="p">):</span>
<span class="k">if</span> <span class="n">response_header</span><span class="o">.</span><span class="n">statusCode</span><span class="p">()</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">301</span><span class="p">,</span> <span class="mi">302</span><span class="p">]:</span>
<span class="n">location</span> <span class="o">=</span> <span class="n">response_header</span><span class="o">.</span><span class="n">value</span><span class="p">(</span><span class="s2">"Location"</span><span class="p">)</span>
<span class="nb">print</span> <span class="s2">"Redirect to: "</span><span class="p">,</span> <span class="n">location</span>
<span class="bp">self</span><span class="o">.</span><span class="n">getId</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">location</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">out</span><span class="p">)</span>
<span class="n">tmp</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">QUrl</span><span class="p">(</span><span class="n">location</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">str</span><span class="p">(</span><span class="n">tmp</span><span class="o">.</span><span class="n">host</span><span class="p">()):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">tmp</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">setHost</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">host</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">port</span><span class="p">(</span><span class="mi">80</span><span class="p">))</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">setPath</span><span class="p">(</span><span class="n">location</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">path</span><span class="p">()</span> <span class="ow">or</span> <span class="s2">"/"</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">out</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">on_req_done</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">error</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Success"</span>
<span class="nb">print</span> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">readAll</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Error"</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">main</span> <span class="o">=</span> <span class="n">MainWidget</span><span class="p">()</span>
<span class="n">main</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</code></pre></div>
<p>运行上面代码, 你将看到下面输出</p>
<div class="highlight"><pre><span></span><code>Redirect to: http://www.linuxzen.com
Success
</code></pre></div>
<h3 id="_4">处理参数</h3>
<p>如果你的 GET 请求的 url 是带着参数传给 <code>QUrl</code> 的, 那么 <code>QUrl.path()</code> 将不会返回带参数的
路径需要自己处理. 可以参见下面的处理函数</p>
<div class="highlight"><pre><span></span><code> <span class="k">def</span> <span class="nf">get_query_string</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">queryPairDelimiter</span><span class="p">()</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
<span class="s2">"</span><span class="si">{0}{1}{2}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">queryValueDelimiter</span><span class="p">(),</span> <span class="n">v</span><span class="p">)</span>
<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">queryItems</span><span class="p">()</span>
<span class="p">)</span>
</code></pre></div>
<p>请求的时候加上就行了</p>
<div class="highlight"><pre><span></span><code> <span class="bp">self</span><span class="o">.</span><span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">path</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"?"</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_query_string</span><span class="p">())</span>
</code></pre></div>
<h2 id="cookie-qnetworkaccessmanager">处理 Cookie ---- QNetworkAccessManager</h2>
<p>QHttp 是无法自动记录 Cookie 和设置 Cookie的, 如果有这个需求就需要 <code>PyQt4.QtNetwork.QNetworkAccessManager</code></p>
<p><code>QNetworkAccessManager</code> 的请求方法不在是传入一个 QString, 而是需要传入一个
<code>PyQt4.QtNetwork.QNetworkRequest</code> 的实例.</p>
<p><code>QNetworkAccessManager</code> 同样是异步的, 可以绑定 <code>QNetworkAccessManager.finished</code> 信号,
这个信号将会给槽传递一个 <code>PyQt4.QtNetwork.QNetworkReply</code> 的实例.</p>
<p><code>QNetworkAccessManager.setCookieJar</code> 可以设置一个 <code>PyQt4.QtNetwork.QNetworkCookieJar</code> 对象来
自动保存和设置 Cookie.</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span><span class="p">,</span> <span class="n">QtNetwork</span>
<span class="k">class</span> <span class="nc">MainWidget</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MainWidget</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">parent</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_cookiejar</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QNetworkCookieJar</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">manager</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QNetworkAccessManager</span><span class="p">(</span><span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">manager</span><span class="o">.</span><span class="n">setCookieJar</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_cookiejar</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">manager</span><span class="o">.</span><span class="n">finished</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">on_reply</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">req</span> <span class="o">=</span> <span class="n">QtNetwork</span><span class="o">.</span><span class="n">QNetworkRequest</span><span class="p">(</span>
<span class="n">QtCore</span><span class="o">.</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"http://www.google.com.hk"</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">manager</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">req</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">on_reply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">reply</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">reply</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cookiejar</span><span class="o">.</span><span class="n">allCookies</span><span class="p">()</span>
<span class="nb">print</span> <span class="n">reply</span><span class="o">.</span><span class="n">rawHeaderList</span><span class="p">()</span>
<span class="c1"># print reply.readAll()</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">widget</span> <span class="o">=</span> <span class="n">MainWidget</span><span class="p">()</span>
<span class="n">widget</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</code></pre></div>
<p>这样就可以携带 Cookie 去请求一些需要 Cookie 认证的请求.</p>
<h2 id="_5">总结</h2>
<p>还有很多内容就不一一介绍, 比如配合 <code>QProgressBar</code>, 添加头部信息, POST 请求, 等.</p>
<p>这里只是简单介绍下 Qt 自己的网络请求处理方法. 在 PyQt 里用 urllib 等库明显不是好的
解决办法, 因为会造成界面卡死. 但是 PyQt4 的方式也实在不是很优雅. 要来来回回的调很多
东西. 但是最起码不会造成界面的卡死.</p>Tornado 多进程实现分析2014-04-11T14:35:00+08:002014-04-11T14:35:00+08:00coldtag:www.linuxzen.com,2014-04-11:/tornado-duo-jin-cheng-shi-xian-fen-xi.html<h2 id="_1">引子</h2>
<p>Tornado 是一个网络异步的的web开发框架, 并且可以利用多进程进行提高效率,
下面是创建一个多进程 tornado 程序的例子.</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 …</span></code></pre></div><h2 id="_1">引子</h2>
<p>Tornado 是一个网络异步的的web开发框架, 并且可以利用多进程进行提高效率,
下面是创建一个多进程 tornado 程序的例子.</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">tornado.web</span>
<span class="kn">import</span> <span class="nn">tornado.httpserver</span>
<span class="kn">import</span> <span class="nn">tornado.ioloop</span>
<span class="kn">import</span> <span class="nn">tornado.netutil</span>
<span class="kn">import</span> <span class="nn">tornado.process</span>
<span class="k">class</span> <span class="nc">LongHandler</span><span class="p">(</span><span class="n">tornado</span><span class="o">.</span><span class="n">web</span><span class="o">.</span><span class="n">RequestHandler</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getpid</span><span class="p">()))</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">tornado</span><span class="o">.</span><span class="n">web</span><span class="o">.</span><span class="n">Application</span><span class="p">(([</span><span class="sa">r</span><span class="s1">'/'</span><span class="p">,</span> <span class="n">LongHandler</span><span class="p">],</span> <span class="p">))</span>
<span class="n">sockets</span> <span class="o">=</span> <span class="n">tornado</span><span class="o">.</span><span class="n">netutil</span><span class="o">.</span><span class="n">bind_sockets</span><span class="p">(</span><span class="mi">8090</span><span class="p">)</span>
<span class="n">tornado</span><span class="o">.</span><span class="n">process</span><span class="o">.</span><span class="n">fork_processes</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="n">server</span> <span class="o">=</span> <span class="n">tornado</span><span class="o">.</span><span class="n">httpserver</span><span class="o">.</span><span class="n">HTTPServer</span><span class="p">(</span><span class="n">app</span><span class="p">)</span>
<span class="n">server</span><span class="o">.</span><span class="n">add_sockets</span><span class="p">(</span><span class="n">sockets</span><span class="p">)</span>
<span class="n">tornado</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<p>上面代码使用 <code>tornado.process.fork_processes</code> 创建了2个子进程, 同时用时访问这个
服务两次, 分别会返回两个相邻的pid. 可以看到 tornado 确实使用了两个进程来同时完成任务.</p>
<p>我一直很好奇 tornado 是如何将请求调度到子进程, 多个子进程又如何不同时处理一个请求呢?</p>
<h2 id="_2">探究</h2>
<p>我们首先是调用 <code>tornado.netutil.bind_sockets</code> 来创建一个 socket(或一个 socket 列表),</p>
<p>接着我们调用 <code>tornado.process.fork_processes</code> 来 fork 子进程,
阅读此函数的代码会发现这个函数仅仅是创建子进程, 然后主进程负责等待子进程, 如果子进
程退出则会根据条件重启子进程, 如果子进程全部退出并不符合重启条件,则主进程退出.</p>
<p>调用这个函数之后, 子进程中函数会返回, 子进程则继续执行调用这个函数之后的代码.</p>
<p>我们在 fork 子进程后做了如下操作.</p>
<div class="highlight"><pre><span></span><code> <span class="n">server</span> <span class="o">=</span> <span class="n">tornado</span><span class="o">.</span><span class="n">httpserver</span><span class="o">.</span><span class="n">HTTPServer</span><span class="p">(</span><span class="n">app</span><span class="p">)</span>
<span class="n">server</span><span class="o">.</span><span class="n">add_sockets</span><span class="p">(</span><span class="n">sockets</span><span class="p">)</span>
<span class="n">tornado</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<p>我们先看看 <code>tornado.httpserver.HTTPServer.add_sockets</code> 发现<code>HTTPServer</code>是继承的
<code>tornado.netutil.TCPServer</code>, add_sockets 也是实现在 <code>TCPServer</code> 中</p>
<p><code>tornado.netutil.TCPServer.add_sockets</code></p>
<div class="highlight"><pre><span></span><code> <span class="k">def</span> <span class="nf">add_sockets</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sockets</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">io_loop</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">io_loop</span> <span class="o">=</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span>
<span class="k">for</span> <span class="n">sock</span> <span class="ow">in</span> <span class="n">sockets</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sockets</span><span class="p">[</span><span class="n">sock</span><span class="o">.</span><span class="n">fileno</span><span class="p">()]</span> <span class="o">=</span> <span class="n">sock</span>
<span class="n">add_accept_handler</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_handle_connection</span><span class="p">,</span>
<span class="n">io_loop</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">io_loop</span><span class="p">)</span>
</code></pre></div>
<p>主要是映射了下 socket 和 socket 对应的文件描述符, 我们看看它调用的
<code>add_accept_handler</code></p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">add_accept_handler</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="n">callback</span><span class="p">,</span> <span class="n">io_loop</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">if</span> <span class="n">io_loop</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">io_loop</span> <span class="o">=</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">accept_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">events</span><span class="p">):</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">connection</span><span class="p">,</span> <span class="n">address</span> <span class="o">=</span> <span class="n">sock</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span>
<span class="k">except</span> <span class="n">socket</span><span class="o">.</span><span class="n">error</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">in</span> <span class="p">(</span><span class="n">errno</span><span class="o">.</span><span class="n">EWOULDBLOCK</span><span class="p">,</span> <span class="n">errno</span><span class="o">.</span><span class="n">EAGAIN</span><span class="p">):</span>
<span class="k">return</span>
<span class="k">raise</span>
<span class="n">callback</span><span class="p">(</span><span class="n">connection</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="n">io_loop</span><span class="o">.</span><span class="n">add_handler</span><span class="p">(</span><span class="n">sock</span><span class="o">.</span><span class="n">fileno</span><span class="p">(),</span> <span class="n">accept_handler</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">)</span>
</code></pre></div>
<p>我们知道 <strong>I/O多路复用</strong> 在处理服务端 socket 时, 当有连接请求过来时, 会触发
可读的事件, 此函数将 socket 在主事件循环中注册读事件(IOLoop.READ), 它的回调
会创建连接, 我注意到回调里的异常捕获有这样几行</p>
<div class="highlight"><pre><span></span><code> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">in</span> <span class="p">(</span><span class="n">errno</span><span class="o">.</span><span class="n">EWOULDBLOCK</span><span class="p">,</span> <span class="n">errno</span><span class="o">.</span><span class="n">EAGAIN</span><span class="p">):</span>
<span class="k">return</span>
<span class="k">raise</span>
</code></pre></div>
<p>发现在创建连接的时候会跳过这个异常呢, 为什么?那么 <code>EWOULDBLOCK</code> 和 <code>EAGAIN</code> 是是什么呢?
通过查找知道它的意思是在非阻塞模式下, 不需要重读或重写, <code>EAGAIN</code> 是 <code>EWOULDBLOCK</code> 在
Windows 上的名字, 所以看到这里就很明确了.</p>
<h2 id="_3">结论</h2>
<p>Tornado 多进程的处理流程是先创建 socket, 然后再 fork 子进程, 这样所有的子进程实际都监听
一个(或多个)文件描述符, 也就是都在监听同样的 socket.</p>
<p>当连接过来所有的子进程都会收到可读事件, 这时候所有的子进程都会跳到 <code>accept_handler</code>
回调函数, 尝试建立连接.</p>
<p>一旦其中一个子进程成功的建立了连接, 当其他子进程再尝试建立这个连接的时候就会触发
<code>EWOULDBLOCK</code>(或 EAGAIN) 错误. 这时候回调函数判断是这个错误则返回函数不做处理.</p>
<p>当成功建立连接的子进程还在处理这个连接的时候又过来一个连接, 这时候就会有另外一个
子进程接手这个连接.</p>
<p>Tornado 就是通过这样一种机制, 利用多进程提升效率, 由于连接只能由一个子进程成功创建,
同一个请求也就不会被多个子进程处理.</p>
<h2 id="_4">后记</h2>
<p>写完才发现, 我所使用的代码是 tornado-2.4.post2 版本, 当前最新代码是 3.3.0,
查看了下最新代码, 最新代码<code>TCPServer</code> 写到单独 <code>tornado.tcpserver</code> 里了, 其他和本文
相关的并没有什么大的变化.</p>记一次 zsh 产生僵尸进程解决2014-03-21T10:47:00+08:002014-03-21T10:47:00+08:00coldtag:www.linuxzen.com,2014-03-21:/ji-yi-ci-zsh-chan-sheng-jiang-shi-jin-cheng-jie-jue.html<h2 id="_1">问题描述</h2>
<p>今天使用 vmplayer 运行了 xp 系统, 关闭后在 zsh 里继续敲命令就阻塞了,
然后就关闭了终端重新打开, 还是阻塞, 重复几次 …</p><h2 id="_1">问题描述</h2>
<p>今天使用 vmplayer 运行了 xp 系统, 关闭后在 zsh 里继续敲命令就阻塞了,
然后就关闭了终端重新打开, 还是阻塞, 重复几次依然如此. 然后使用 gVim
将 shell 切换到 bash, 终端可以正常打开, 然后运行</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>ps<span class="w"> </span>aux<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27552</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.1<span class="w"> </span><span class="m">47244</span><span class="w"> </span><span class="m">5164</span><span class="w"> </span>?<span class="w"> </span>Ss<span class="w"> </span><span class="m">09</span>:38<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27553</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.1<span class="w"> </span><span class="m">47244</span><span class="w"> </span><span class="m">5156</span><span class="w"> </span>?<span class="w"> </span>Ss<span class="w"> </span><span class="m">09</span>:38<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27600</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">47348</span><span class="w"> </span><span class="m">3492</span><span class="w"> </span>?<span class="w"> </span>D<span class="w"> </span><span class="m">09</span>:38<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27609</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">47348</span><span class="w"> </span><span class="m">3488</span><span class="w"> </span>?<span class="w"> </span>D<span class="w"> </span><span class="m">09</span>:38<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27614</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">47348</span><span class="w"> </span><span class="m">3484</span><span class="w"> </span>?<span class="w"> </span>D<span class="w"> </span><span class="m">09</span>:38<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>zsh
wh<span class="w"> </span><span class="m">27697</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.1<span class="w"> </span><span class="m">47248</span><span class="w"> </span><span class="m">5172</span><span class="w"> </span>?<span class="w"> </span>Ss<span class="w"> </span><span class="m">09</span>:39<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>-/bin/zsh
wh<span class="w"> </span><span class="m">27718</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">47356</span><span class="w"> </span><span class="m">3496</span><span class="w"> </span>?<span class="w"> </span>D<span class="w"> </span><span class="m">09</span>:39<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>-/bin/zsh
root<span class="w"> </span><span class="m">28040</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">36640</span><span class="w"> </span><span class="m">2812</span><span class="w"> </span>tty2<span class="w"> </span>Ss+<span class="w"> </span><span class="m">09</span>:40<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>-zsh
wh<span class="w"> </span><span class="m">28628</span><span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">47356</span><span class="w"> </span><span class="m">3492</span><span class="w"> </span>?<span class="w"> </span>D<span class="w"> </span><span class="m">09</span>:42<span class="w"> </span><span class="m">0</span>:00<span class="w"> </span>-/bin/zsh
</code></pre></div>
<p><s>发现好多僵尸进程, 而且都 kill 不掉.</s>
发现好多 D 状态的进程, 查看 <code>ps</code> 的手册, D 状态的解释是</p>
<div class="highlight"><pre><span></span><code>D uninterruptible sleep (usually io)
</code></pre></div>
<p>进程可能是由 I/O 引起的不可间断的等待.</p>
<h2 id="_2">解决</h2>
<p>首先当然是 <code>strace</code> 登场</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>strace<span class="w"> </span>zsh
...
pipe<span class="o">([</span><span class="m">3</span>,<span class="w"> </span><span class="m">4</span><span class="o">])</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
fcntl<span class="o">(</span><span class="m">3</span>,<span class="w"> </span>F_DUPFD,<span class="w"> </span><span class="m">10</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">11</span>
close<span class="o">(</span><span class="m">3</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
fcntl<span class="o">(</span><span class="m">4</span>,<span class="w"> </span>F_DUPFD,<span class="w"> </span><span class="m">10</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">12</span>
close<span class="o">(</span><span class="m">4</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
rt_sigprocmask<span class="o">(</span>SIG_BLOCK,<span class="w"> </span><span class="o">[</span>CHLD<span class="o">]</span>,<span class="w"> </span><span class="o">[</span>CHLD<span class="w"> </span>WINCH<span class="o">]</span>,<span class="w"> </span><span class="m">8</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
clone<span class="o">(</span><span class="nv">child_stack</span><span class="o">=</span><span class="m">0</span>,<span class="w"> </span><span class="nv">flags</span><span class="o">=</span>CLONE_CHILD_CLEARTID<span class="p">|</span>CLONE_CHILD_SETTID<span class="p">|</span>SIGCHLD,<span class="w"> </span><span class="nv">child_tidptr</span><span class="o">=</span>0x7fd94caef9d0<span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">8750</span>
close<span class="o">(</span><span class="m">12</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
fcntl<span class="o">(</span><span class="m">11</span>,<span class="w"> </span>F_GETFL<span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">(</span>flags<span class="w"> </span>O_RDONLY<span class="o">)</span>
fstat<span class="o">(</span><span class="m">11</span>,<span class="w"> </span><span class="o">{</span><span class="nv">st_mode</span><span class="o">=</span>S_IFIFO<span class="p">|</span><span class="m">0600</span>,<span class="w"> </span><span class="nv">st_size</span><span class="o">=</span><span class="m">0</span>,<span class="w"> </span>...<span class="o">})</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
mmap<span class="o">(</span>NULL,<span class="w"> </span><span class="m">4096</span>,<span class="w"> </span>PROT_READ<span class="p">|</span>PROT_WRITE,<span class="w"> </span>MAP_PRIVATE<span class="p">|</span>MAP_ANONYMOUS,<span class="w"> </span>-1,<span class="w"> </span><span class="m">0</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>0x7fd94caf3000
lseek<span class="o">(</span><span class="m">11</span>,<span class="w"> </span><span class="m">0</span>,<span class="w"> </span>SEEK_CUR<span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-1<span class="w"> </span>ESPIPE<span class="w"> </span><span class="o">(</span>Illegal<span class="w"> </span>seek<span class="o">)</span>
read<span class="o">(</span><span class="m">11</span>,
</code></pre></div>
<p>从上面可以看出, zsh 打开了一个 pipe, 但是读取的时候阻塞了.</p>
<p>现在调用 <code>strace -f</code> 查看管道另一头的在做些什么</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>strace<span class="w"> </span>-f<span class="w"> </span>zsh
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>chdir<span class="o">(</span><span class="s2">".."</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>stat<span class="o">(</span><span class="s2">".."</span>,<span class="w"> </span><span class="o">{</span><span class="nv">st_mode</span><span class="o">=</span>S_IFDIR<span class="p">|</span><span class="m">0755</span>,<span class="w"> </span><span class="nv">st_size</span><span class="o">=</span><span class="m">4096</span>,<span class="w"> </span>...<span class="o">})</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>openat<span class="o">(</span>AT_FDCWD,<span class="w"> </span><span class="s2">".."</span>,<span class="w"> </span>O_RDONLY<span class="p">|</span>O_NONBLOCK<span class="p">|</span>O_DIRECTORY<span class="p">|</span>O_CLOEXEC<span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>getdents<span class="o">(</span><span class="m">3</span>,<span class="w"> </span>/*<span class="w"> </span><span class="m">12</span><span class="w"> </span>entries<span class="w"> </span>*/,<span class="w"> </span><span class="m">32768</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">344</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>lstat<span class="o">(</span><span class="s2">"../home"</span>,<span class="w"> </span><span class="o">{</span><span class="nv">st_mode</span><span class="o">=</span>S_IFDIR<span class="p">|</span><span class="m">0755</span>,<span class="w"> </span><span class="nv">st_size</span><span class="o">=</span><span class="m">4096</span>,<span class="w"> </span>...<span class="o">})</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>close<span class="o">(</span><span class="m">3</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>chdir<span class="o">(</span><span class="s2">".."</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>stat<span class="o">(</span><span class="s2">".."</span>,<span class="w"> </span><span class="o">{</span><span class="nv">st_mode</span><span class="o">=</span>S_IFDIR<span class="p">|</span><span class="m">0755</span>,<span class="w"> </span><span class="nv">st_size</span><span class="o">=</span><span class="m">4096</span>,<span class="w"> </span>...<span class="o">})</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>openat<span class="o">(</span>AT_FDCWD,<span class="w"> </span><span class="s2">".."</span>,<span class="w"> </span>O_RDONLY<span class="p">|</span>O_NONBLOCK<span class="p">|</span>O_DIRECTORY<span class="p">|</span>O_CLOEXEC<span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>getdents<span class="o">(</span><span class="m">3</span>,<span class="w"> </span>/*<span class="w"> </span><span class="m">26</span><span class="w"> </span>entries<span class="w"> </span>*/,<span class="w"> </span><span class="m">32768</span><span class="o">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">672</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>lstat<span class="o">(</span><span class="s2">"../lib"</span>,<span class="w"> </span><span class="o">{</span><span class="nv">st_mode</span><span class="o">=</span>S_IFLNK<span class="p">|</span><span class="m">0777</span>,<span class="w"> </span><span class="nv">st_size</span><span class="o">=</span><span class="m">7</span>,<span class="w"> </span>...<span class="o">})</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span>
<span class="o">[</span>pid<span class="w"> </span><span class="m">28429</span><span class="o">]</span><span class="w"> </span>lstat<span class="o">(</span><span class="s2">"../mnt"</span>,
</code></pre></div>
<p>上面可以看到 zsh 在 <code>lstat("../mnt")</code> 的时候产生了等待</p>
<p>/mnt 一般挂载一些东西, 运行 <code>df -h</code> 查看一下, 发现也很慢, 当然 strace 下 它</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>strace<span class="w"> </span>df<span class="w"> </span>-h
...
stat<span class="o">(</span><span class="s2">"/mnt"</span>,
</code></pre></div>
<p>发现它同样卡在了读取 <code>/mnt</code>. 尝试 <code>umount</code> 它:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>sudo<span class="w"> </span>umount<span class="w"> </span>/mnt
umount:<span class="w"> </span>/mnt:<span class="w"> </span>target<span class="w"> </span>is<span class="w"> </span>busy
<span class="w"> </span><span class="o">(</span>In<span class="w"> </span>some<span class="w"> </span>cases<span class="w"> </span>useful<span class="w"> </span>info<span class="w"> </span>about<span class="w"> </span>processes<span class="w"> </span>that
<span class="w"> </span>use<span class="w"> </span>the<span class="w"> </span>device<span class="w"> </span>is<span class="w"> </span>found<span class="w"> </span>by<span class="w"> </span>lsof<span class="o">(</span><span class="m">8</span><span class="o">)</span><span class="w"> </span>or<span class="w"> </span>fuser<span class="o">(</span><span class="m">1</span><span class="o">)</span>.<span class="o">)</span>
</code></pre></div>
<p>说是被某个程序使用, 现在该 <code>lsof</code> 登场了</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>lsof<span class="w"> </span>/mnt
lsof<span class="w"> </span>/mnt
lsof:<span class="w"> </span>WARNING:<span class="w"> </span>can<span class="err">'</span>t<span class="w"> </span>stat<span class="o">()</span><span class="w"> </span>cifs<span class="w"> </span>file<span class="w"> </span>system<span class="w"> </span>/mnt
<span class="w"> </span>Output<span class="w"> </span>information<span class="w"> </span>may<span class="w"> </span>be<span class="w"> </span>incomplete.
lsof:<span class="w"> </span>status<span class="w"> </span>error<span class="w"> </span>on<span class="w"> </span>/mnt:<span class="w"> </span>Host<span class="w"> </span>is<span class="w"> </span>down
</code></pre></div>
<p>好吧, 看到这里我就想明白了, 因为开虚拟机的时候我挂载了虚拟机里的一个共享目录</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>sudo<span class="w"> </span>mount<span class="w"> </span>-t<span class="w"> </span>cifs<span class="w"> </span>-o<span class="w"> </span>guest<span class="w"> </span>//ip.of.host/share<span class="w"> </span>/mnt
</code></pre></div>
<p>在虚拟机关闭之后没有卸载, 导致在对这个目录 <code>stat</code> 的时候会有一个网络超时时间.</p>
<p>然后就卸载它</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>umount<span class="w"> </span>-a<span class="w"> </span>-t<span class="w"> </span>cifs<span class="w"> </span>-l<span class="w"> </span>/mnt
</code></pre></div>
<h2 id="_3">结果</h2>
<p>现在打开 zsh 也不阻塞了, <code>df -h</code> 也同样不阻塞了. 上面 Ss 的进程可以用 -9 杀掉,
但是 D 状态进程无法杀掉.</p>
<h2 id="_4">参考</h2>
<ul>
<li>lsof: <a href="http://www.ibm.com/developerworks/cn/aix/library/au-lsof.html">http://stackoverflow.com/questions/74626/how-do-you-force-a-cifs-connection-to-unmount</a></li>
<li>卸载 cifs: <a href="http://stackoverflow.com/questions/74626/how-do-you-force-a-cifs-connection-to-unmount">http://stackoverflow.com/questions/74626/how-do-you-force-a-cifs-connection-to-unmount</a></li>
</ul>
<h2 id="_5">更新</h2>
<p>经过 <a href="http://lilydjwg.is-programmer.com/">依云</a> 和 <a href="http://eleveni386.7axu.com">eleven</a>
的指正对文章做了一些修改, 感谢两位.</p>解决一直崩溃的 Adobe Flash Player2014-03-12T00:00:00+08:002014-03-12T00:00:00+08:00coldtag:www.linuxzen.com,2014-03-12:/jie-jue-yi-zhi-beng-kui-de-adobe-flash-player.html<p>1月份的时候决定从 Ubuntu 换到 Archlinux, 换完之后 Flash Player 就一直没正常过.
一打开视屏就 crash. 从那之后就一直用手机看视屏, 很别扭, 放着大屏不用一直盯 …</p><p>1月份的时候决定从 Ubuntu 换到 Archlinux, 换完之后 Flash Player 就一直没正常过.
一打开视屏就 crash. 从那之后就一直用手机看视屏, 很别扭, 放着大屏不用一直盯着手机看看.
今天下定决心找找原因. google 了一阵也没有啥结果, 所以我决定卸载现有的, 手动安装一个试试.</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>sudo<span class="w"> </span>pacman<span class="w"> </span>-R<span class="w"> </span>flashplugin
</code></pre></div>
<p>结果卸载的时候输出段信息:</p>
<div class="highlight"><pre><span></span><code>warning: /etc/adobe/mms.cfg saved as /etc/adobe/mms.cfg.pacsave
</code></pre></div>
<p>然后我看了下文件内容</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>cat<span class="w"> </span>/etc/adobe/mms.cfg.pacsave
<span class="c1">#Hardware video decoding</span>
<span class="nv">EnableLinuxHWVideoDecode</span><span class="o">=</span><span class="m">1</span>
</code></pre></div>
<p>瞬间觉得可能是这个选项引起的, 所以我又装上了 flashplugin</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>sudo<span class="w"> </span>pacman<span class="w"> </span>-S<span class="w"> </span>flashplugin
</code></pre></div>
<p>同样看到一段信息</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="o">>></span><span class="w"> </span>
<span class="w"> </span><span class="o">>></span><span class="w"> </span><span class="k">If</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="n">have</span><span class="w"> </span><span class="n">an</span><span class="w"> </span><span class="n">NVIDIA</span><span class="w"> </span><span class="n">card</span><span class="w"> </span><span class="n">that</span><span class="w"> </span><span class="n">supports</span><span class="w"> </span><span class="n">libvdpau</span><span class="w"> </span><span class="ow">or</span><span class="w"> </span><span class="n">Broadcom</span><span class="w"> </span><span class="n">Crystal</span><span class="w"> </span><span class="n">HD</span><span class="w"> </span><span class="n">chips</span><span class="p">,</span>
<span class="w"> </span><span class="o">>></span><span class="w"> </span><span class="n">uncomment</span><span class="w"> </span><span class="n">EnableLinuxHWVideoDecode</span><span class="o">=</span><span class="mi">1</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">adobe</span><span class="o">/</span><span class="n">mms</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span>
<span class="w"> </span><span class="o">>></span><span class="w"> </span><span class="k">If</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="k">into</span><span class="w"> </span><span class="n">problems</span><span class="p">,</span><span class="w"> </span><span class="n">please</span><span class="w"> </span><span class="n">contact</span><span class="w"> </span><span class="n">nVidia</span><span class="w"> </span><span class="ow">or</span><span class="w"> </span><span class="n">Broadcom</span><span class="w"> </span><span class="n">along</span><span class="w"> </span><span class="k">with</span><span class="w"> </span><span class="n">your</span><span class="w"> </span><span class="k">system</span><span class="w"> </span><span class="n">config</span><span class="w"> </span><span class="n">info</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">driver</span><span class="w"> </span><span class="n">version</span><span class="p">.</span>
<span class="w"> </span><span class="o">>></span><span class="w"> </span>
<span class="n">Optional</span><span class="w"> </span><span class="n">dependencies</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">flashplugin</span>
<span class="w"> </span><span class="nl">libvdpau</span><span class="p">:</span><span class="w"> </span><span class="n">GPU</span><span class="w"> </span><span class="n">acceleration</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="n">Nvidia</span><span class="w"> </span><span class="n">card</span><span class="w"> </span><span class="o">[</span><span class="n">installed</span><span class="o">]</span>
</code></pre></div>
<p>然后我就知道了为什么会出现那个选项打开的情况了, 一开始安装 Archlinux 的时候我误以为我的显卡是 NVIDIA 的,
然后就安装了 NVIDIA 的驱动, 后来仔细一看原来是集成的 Intel 显卡XD, 估计在一开始安装 flashplugin
的时候自动开启了. 然后换了显卡驱动我也不清楚.现在看新的配置文件 EnableLinuxHWVIdeoDecode=1 是注释掉的.</p>
<p>接下来就是验证喽, 打开一个视屏, 果然不 crash 了.</p>
<p>然后开启这个选项, 果然预料中的接着 crash.</p>使用 Pygments 对 Vimwiki 进行代码高亮2013-12-27T00:00:00+08:002013-12-27T00:00:00+08:00coldtag:www.linuxzen.com,2013-12-27:/shi-yong-pygments-dui-vimwiki-jin-xing-dai-ma-gao-liang.html<p>Vimwiki 推荐的代码高亮机制是通过一个 JavaScript 插件来完成的, 那样需要加载很多 js,
所以不想使用, 比较倾向使用 Pygments 在 Vimwiki 生成 HTML 的 …</p><p>Vimwiki 推荐的代码高亮机制是通过一个 JavaScript 插件来完成的, 那样需要加载很多 js,
所以不想使用, 比较倾向使用 Pygments 在 Vimwiki 生成 HTML 的时候对代码进行高亮.</p>
<h1 id="_1">尝试</h1>
<h2 id="custom_wiki2html">使用 custom_wiki2html 选项</h2>
<p>仔细的看了 Vimwiki 的帮助文档, 发现有一个 custom_wiki2html
(<code>:h vimwiki-option-custom_wiki2html</code>) 的选项可以指定自己
的脚本来处理 wiki2html, 尝试了一下, 发现这个脚本是在生成 HTML 之前调用,
而且如果对 wiki 文件处理之后无法替换回原来的内容(后来发现这个仅仅是对使用 Markdown
语法作为 Wiki 语法设定的), 所以放弃了.</p>
<h2 id="fork">Fork 仓库, 更改代码</h2>
<p>后来想想既然原生的没有解决办法, 所以就干脆在 github 上 fork 了仓库
clone 到本地进行修改, 通过直接 hack 代码在 Vimwiki 处理之前对代码进行高亮.</p>
<h3 id="_2">思路</h3>
<p>粗略的看了下代码, Vimwiki 是将文件读入, 然后逐行处理, 看来只能在文件读取之后
对内容做一些操作.</p>
<p>Vimscript 的 readfile 返回一个列表, 每一个元素代表一行, 编写一个函数处理这个
列表, 并返回, 在 autoload/vimwiki/html.vim 里的 第 1350 行找到
<code>vimwiki#html#CustomWiki2HTML</code> 函数, 并在其上面添加一个函数</p>
<div class="highlight"><pre><span></span><code><span class="k">function</span><span class="p">!</span> s:highlight_code_with_pygments<span class="p">(</span>lsource<span class="p">)</span> <span class="c">"{{{</span>
<span class="k">if</span> <span class="p">!</span>has<span class="p">(</span><span class="s1">'python'</span><span class="p">)</span>
<span class="k">return</span> <span class="k">a</span>:lsource
<span class="k">endif</span>
<span class="k">let</span> s:lsource <span class="p">=</span> deepcopy<span class="p">(</span><span class="k">a</span>:lsource<span class="p">)</span>
<span class="k">let</span> s:content <span class="p">=</span> <span class="s1">''</span>
<span class="k">python</span> <span class="o"><<</span>EOF
<span class="k">def</span> <span class="nf">handle</span><span class="p">():</span>
<span class="kn">import</span> <span class="nn">vim</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pygments</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s2">"echoerr 'Cannot import pygments library, please install it.'"</span><span class="p">)</span>
<span class="k">return</span>
<span class="kn">from</span> <span class="nn">pygments.lexers</span> <span class="kn">import</span> <span class="n">get_lexer_by_name</span>
<span class="kn">from</span> <span class="nn">pygments.formatters</span> <span class="kn">import</span> <span class="n">HtmlFormatter</span>
<span class="kn">from</span> <span class="nn">pygments</span> <span class="kn">import</span> <span class="n">highlight</span>
<span class="kn">from</span> <span class="nn">pygments.util</span> <span class="kn">import</span> <span class="n">ClassNotFound</span>
<span class="n">CODE_RE</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s1">'\n({{{(\w*?)\s(.*?)\s}}})'</span><span class="p">,</span> <span class="n">re</span><span class="o">.</span><span class="n">M</span><span class="o">|</span><span class="n">re</span><span class="o">.</span><span class="n">U</span><span class="o">|</span><span class="n">re</span><span class="o">.</span><span class="n">S</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">vim</span><span class="o">.</span><span class="n">eval</span><span class="p">(</span><span class="s2">"s:lsource"</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">new</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">css_class</span> <span class="o">=</span> <span class="n">vim</span><span class="o">.</span><span class="n">eval</span><span class="p">(</span><span class="s2">"VimwikiGet('pygments_class')"</span><span class="p">)</span>
<span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">lang_type</span><span class="p">,</span> <span class="n">code</span> <span class="ow">in</span> <span class="n">CODE_RE</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">content</span><span class="p">):</span>
<span class="n">lang_type</span> <span class="o">=</span> <span class="n">lang_type</span> <span class="ow">or</span> <span class="s2">"text"</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">lexer</span> <span class="o">=</span> <span class="n">get_lexer_by_name</span><span class="p">(</span><span class="n">lang_type</span><span class="p">)</span>
<span class="k">except</span> <span class="n">ClassNotFound</span><span class="p">:</span>
<span class="n">lexer</span> <span class="o">=</span> <span class="n">get_lexer_by_name</span><span class="p">(</span><span class="s2">"text"</span><span class="p">)</span>
<span class="n">formatter</span> <span class="o">=</span> <span class="n">HtmlFormatter</span><span class="p">(</span><span class="n">encoding</span><span class="o">=</span><span class="s2">"utf8"</span><span class="p">,</span> <span class="n">cssclass</span><span class="o">=</span><span class="n">css_class</span><span class="p">,</span>
<span class="n">noclasses</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">style</span><span class="o">=</span><span class="s2">"default"</span><span class="p">,</span>
<span class="n">linenos</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">nowrap</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span>
<span class="n">hcode</span> <span class="o">=</span> <span class="n">highlight</span><span class="p">(</span><span class="n">code</span><span class="p">,</span> <span class="n">lexer</span><span class="p">,</span> <span class="n">formatter</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">hcode</span><span class="p">)</span>
<span class="k">if</span> <span class="n">new</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
<span class="n">new</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">if</span> <span class="n">new</span><span class="p">:</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s2">"let s:content='</span><span class="si">%s</span><span class="s2">'"</span> <span class="o">%</span> <span class="n">content</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"'"</span><span class="p">,</span> <span class="s2">"</span><span class="se">\'</span><span class="s2">"</span><span class="p">))</span>
<span class="n">handle</span><span class="p">()</span>
EOF
<span class="k">if</span> s:content <span class="p">!=</span> <span class="s1">''</span>
<span class="k">let</span> s:lsource <span class="p">=</span> split<span class="p">(</span>s:content<span class="p">,</span> <span class="s1">'\n'</span><span class="p">)</span>
<span class="k">endif</span>
<span class="k">return</span> s:lsource
<span class="k">endfunction</span> <span class="c">"}}}</span>
</code></pre></div>
<p>然后在 <code>vimwiki#html#Wiki2HTML</code> 函数内读取文件的下面调用此函数</p>
<div class="highlight"><pre><span></span><code>...
<span class="k">let</span> lsource <span class="p">=</span> readfile<span class="p">(</span>wikifile<span class="p">)</span>
<span class="k">let</span> lsource <span class="p">=</span> s:highlight_code_with_pygments<span class="p">(</span>lsource<span class="p">)</span>
...
</code></pre></div>
<p>当然需要还需要拷贝 css 文件, 修改模板等, 这里不一一详述</p>
<h3 id="_3">结果</h3>
<p>好吧, 确实工作了, 但是新的问题出来了, 高亮后替换成HTML的内容会被 Vimwiki 第二次处理
所以 HTML 的格式全乱了, CSS 样式也肯定不行了. 找了一下没有办法能避免 Vimwiki 不
处理一段内容(为什么已经是 HTMl 了 Vimwiki 还要处理呢)</p>
<p>上述方法不行, 那只能再想想办法了</p>
<h3 id="nowiki">使用特殊的注释: %nowiki</h3>
<p>为了解决上面重复处理已转化成 HTML 内容的问题, 我添加 %nowiki 语法,
以 %nowiki 这个开头的行将不处理. 然后将高亮的代码替换成 %nowiki 开始的行,
将上面 Python 相关代码加上如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="o">...</span>
<span class="n">hcode</span> <span class="o">=</span> <span class="n">highlight</span><span class="p">(</span><span class="n">code</span><span class="p">,</span> <span class="n">lexer</span><span class="p">,</span> <span class="n">formatter</span><span class="p">)</span>
<span class="c1"># 添加以下语句</span>
<span class="n">hcode</span> <span class="o">=</span> <span class="s2">"%nowiki"</span> <span class="o">+</span> <span class="s1">'</span><span class="se">\n</span><span class="s1">%nowiki'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">hcode</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">))</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">hcode</span><span class="p">)</span>
<span class="o">..</span>
</code></pre></div>
<p>然后找到 <code>s:parse_line</code> 函数, 在处理 toc 的下面添加</p>
<div class="highlight"><pre><span></span><code><span class="c"> " toc -- placeholder "{{{</span>
<span class="k">if</span> <span class="p">!</span>processed
<span class="k">if</span> line <span class="p">=~</span> <span class="s1">'^\s*%toc'</span>
<span class="k">let</span> processed <span class="p">=</span> <span class="m">1</span>
<span class="k">let</span> param <span class="p">=</span> matchstr<span class="p">(</span>line<span class="p">,</span> <span class="s1">'^\s*%toc\s\zs.*'</span><span class="p">)</span>
<span class="k">let</span> state.placeholder <span class="p">=</span> [<span class="s1">'toc'</span><span class="p">,</span> param]
<span class="k">endif</span>
<span class="k">endif</span>
<span class="c"> "}}}</span>
<span class="c"> " 添加处理 nowiki</span>
<span class="c"> " nowiki "{{{</span>
<span class="k">if</span> <span class="p">!</span>processed
<span class="k">if</span> line<span class="p">=~</span> <span class="s1">'^\s*%nowiki'</span>
<span class="k">let</span> processed <span class="p">=</span> <span class="m">1</span>
<span class="k">call</span> add<span class="p">(</span>res_lines<span class="p">,</span> substitute<span class="p">(</span>line<span class="p">,</span> <span class="s1">'^\s*%nowiki'</span><span class="p">,</span> <span class="s1">''</span><span class="p">,</span> <span class="s1">''</span><span class="p">))</span>
<span class="k">endif</span>
<span class="k">endif</span> <span class="c">"}}}</span>
</code></pre></div>
<p>然后找到 <code>s:safe_html</code> 函数, 添加如下语句</p>
<div class="highlight"><pre><span></span><code><span class="k">function</span><span class="p">!</span> s:safe_html<span class="p">(</span>line<span class="p">)</span> <span class="c">"{{{</span>
<span class="c"> " 不处理 %nowiki </span>
<span class="k">if</span> <span class="k">a</span>:line<span class="p">=~</span> <span class="s1">'^\s*%nowiki'</span>
<span class="k">return</span> <span class="k">a</span>:line
<span class="k">endif</span>
...
<span class="k">endfunction</span>
</code></pre></div>
<p>大功告成, 这样就可以正常的使用 pygments 进行高亮代码</p>
<h1 id="_4">成果</h1>
<p>修改后的代码放在了 github 上, 在 <a href="https://github.com/coldnight/vimwiki">这里</a>,
有需要的可以安装, 但这个更改仅仅是对我个人使用, 可能不会跟随主线程版本的更新.</p>
<p>当然我添加了一些选项来开启/关闭使用pygments, 安装后在 Vim 中执行 <code>:h vimwiki-option-use_pygments</code></p>
<p>大家通过代码也看到了, 要使用改变之后的插件 Vim 需要编译 +python 并且需要安装 pygments Python 库.</p>使用 Vimwiki + git 做知识管理2013-12-26T00:00:00+08:002013-12-26T00:00:00+08:00coldtag:www.linuxzen.com,2013-12-26:/shi-yong-vimwiki-git-zuo-zhi-shi-guan-li.html<p>一直在找一个合适的知识管理工具, 用过 Evernote, 但是不支持 Markdown, 用了一段时间也放弃了.
最近 python-cn 列表里也在讨论这个问题, 看到 …</p><p>一直在找一个合适的知识管理工具, 用过 Evernote, 但是不支持 Markdown, 用了一段时间也放弃了.
最近 python-cn 列表里也在讨论这个问题, 看到有人使用 Vimwiki, 所以就尝试了一下.</p>
<p>安装后,试着写了点东西, 发现很方便做知识管理和记录笔记, 可以生成HTML, 可以定制模板,
这里不讨论如何使用, Vimwiki 的文档介绍的很详细,</p>
<p>我使用<a href="http://bootcss.com">bootstrap</a>和 jquery 对模板进行了一些定制:</p>
<ul>
<li>添加导航</li>
<li>将toc移动到左侧</li>
</ul>
<p>下面将介绍我是如何做的, 并在最后附上如何部署的</p>
<h3 id="_1">指定模板</h3>
<p>首先需要更改默认模板</p>
<div class="highlight"><pre><span></span><code> <span class="k">let</span> <span class="k">g</span>:vimwiki_list <span class="p">=</span> [{<span class="s1">'path'</span>: <span class="s1">'~/vimwiki'</span><span class="p">,</span>
\ <span class="s1">'path_html'</span>: <span class="s1">'~/vimwiki_html'</span><span class="p">,</span>
\ <span class="s1">'template_path'</span>: <span class="s1">'~/vimwiki/template'</span><span class="p">,</span>
\ <span class="s1">'template_default'</span>: <span class="s2">"default.tpl"</span>}]
</code></pre></div>
<p>并将默认的模板作为模板进行修改</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>-p<span class="w"> </span>~/vimwiki/template
cp<span class="w"> </span>~/.vim/bundle/vimwiki/autoload/vimwiki/default.tpl<span class="w"> </span>~/vimwiki/template
</code></pre></div>
<h3 id="_2">添加静态文件</h3>
<p>将jqeury和bootstrap的js和css文件放到 ~/vimwik_html目录下,</p>
<p>为了统一修改和发布 Wiki, 我将静态文件放在 ~/vimwiki 的 static 目录下,
并在 ~/vimwiki_html 创建一个软链链接到这个目录, 下面是我 static 目录的一个快照</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>tree<span class="w"> </span>~/vimwiki/static<span class="w"> </span>
~/vimwiki/static
├──<span class="w"> </span>bootstrap
│<span class="w"> </span>├──<span class="w"> </span>css
│<span class="w"> </span>│<span class="w"> </span>├──<span class="w"> </span>bootstrap.min.css
│<span class="w"> </span>│<span class="w"> </span>└──<span class="w"> </span>bootstrap-theme.min.css
│<span class="w"> </span>├──<span class="w"> </span>fonts
│<span class="w"> </span>│<span class="w"> </span>├──<span class="w"> </span>glyphicons-halflings-regular.eot
│<span class="w"> </span>│<span class="w"> </span>├──<span class="w"> </span>glyphicons-halflings-regular.svg
│<span class="w"> </span>│<span class="w"> </span>├──<span class="w"> </span>glyphicons-halflings-regular.ttf
│<span class="w"> </span>│<span class="w"> </span>└──<span class="w"> </span>glyphicons-halflings-regular.woff
│<span class="w"> </span>└──<span class="w"> </span>js
│<span class="w"> </span>└──<span class="w"> </span>bootstrap.min.js
├──<span class="w"> </span>css
│<span class="w"> </span>└──<span class="w"> </span>wiki.css
└──<span class="w"> </span>js
<span class="w"> </span>├──<span class="w"> </span>jquery-1.8.3.min.js
<span class="w"> </span>└──<span class="w"> </span>wiki.js
<span class="m">6</span><span class="w"> </span>directories,<span class="w"> </span><span class="m">10</span><span class="w"> </span>files
</code></pre></div>
<p>其中<code>wiki.js</code>和<code>wiki.css</code>分别是自定义的Javascript和CSS
然后在~/vimwiki_html中创建链接</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>ln<span class="w"> </span>-s<span class="w"> </span>~/vimwiki/static<span class="w"> </span>~/vimwiki_html/static
</code></pre></div>
<h3 id="_3">在模板中引用静态文件</h3>
<p>结合 <code>%root_path%</code>模板变量引入静态文件</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">link</span> <span class="na">rel</span><span class="o">=</span><span class="s">"Stylesheet"</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/css"</span> <span class="na">href</span><span class="o">=</span><span class="s">"%root_path%static/css/wiki.css"</span><span class="p">></span>
<span class="p"><</span><span class="nt">link</span> <span class="na">rel</span><span class="o">=</span><span class="s">"Stylesheet"</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/css"</span> <span class="na">href</span><span class="o">=</span><span class="s">"%root_path%static/bootstrap/css/bootstrap.min.css"</span><span class="p">></span>
<span class="p"><</span><span class="nt">script</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/javascript"</span> <span class="na">src</span><span class="o">=</span><span class="s">"%root_path%static/js/jquery-1.8.3.min.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
<span class="p"><</span><span class="nt">script</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/javascript"</span> <span class="na">src</span><span class="o">=</span><span class="s">"%root_path%static/bootstrap/js/bootstrap.min.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
<span class="p"><</span><span class="nt">script</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/javascript"</span> <span class="na">src</span><span class="o">=</span><span class="s">"%root_path%static/js/wiki.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
</code></pre></div>
<h3 id="_4">创建导航</h3>
<p>根据 Boostrap 文档中的例子很容易就可以创建一个好看的导航</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">nav</span> <span class="na">class</span><span class="o">=</span><span class="s">"navbar navbar-default navbar-inverse"</span> <span class="na">role</span><span class="o">=</span><span class="s">"navigation"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"container"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"navbar-header"</span><span class="p">></span>
<span class="p"><</span><span class="nt">button</span> <span class="na">data-target</span><span class="o">=</span><span class="s">".bs-navbar-collapse"</span> <span class="na">data-toggle</span><span class="o">=</span><span class="s">"collapse"</span> <span class="na">type</span><span class="o">=</span><span class="s">"button"</span> <span class="na">class</span><span class="o">=</span><span class="s">"navbar-toggle"</span><span class="p">></span>
<span class="p"><</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">"sr-only"</span><span class="p">></</span><span class="nt">span</span><span class="p">></span>
<span class="p"><</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">"icon-bar"</span><span class="p">></</span><span class="nt">span</span><span class="p">></span>
<span class="p"><</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">"icon-bar"</span><span class="p">></</span><span class="nt">span</span><span class="p">></span>
<span class="p"><</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">"icon-bar"</span><span class="p">></</span><span class="nt">span</span><span class="p">></span>
<span class="p"></</span><span class="nt">button</span><span class="p">></span>
<span class="p"><</span><span class="nt">a</span> <span class="na">class</span><span class="o">=</span><span class="s">"navbar-brand"</span> <span class="na">href</span><span class="o">=</span><span class="s">"/index.html"</span><span class="p">></span>cold's wiki<span class="p"></</span><span class="nt">a</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"collapse navbar-collapse"</span><span class="p">></span>
<span class="p"><</span><span class="nt">ul</span> <span class="na">class</span><span class="o">=</span><span class="s">"nav navbar-nav"</span><span class="p">></span>
<span class="p"><</span><span class="nt">li</span><span class="p">><</span><span class="nt">a</span> <span class="na">href</span><span class="o">=</span><span class="s">"%root_path%index.html"</span><span class="p">></span>首页<span class="p"></</span><span class="nt">a</span><span class="p">></</span><span class="nt">li</span><span class="p">></span>
<span class="p"><</span><span class="nt">li</span><span class="p">><</span><span class="nt">a</span> <span class="na">href</span><span class="o">=</span><span class="s">"%root_path%diary/diary.html"</span><span class="p">></span>日记<span class="p"></</span><span class="nt">a</span><span class="p">></</span><span class="nt">li</span><span class="p">></span>
<span class="p"><</span><span class="nt">li</span><span class="p">><</span><span class="nt">a</span> <span class="na">href</span><span class="o">=</span><span class="s">"%root_path%TODO.html"</span><span class="p">></span>TODO<span class="p"></</span><span class="nt">a</span><span class="p">></</span><span class="nt">li</span><span class="p">></span>
<span class="p"></</span><span class="nt">ul</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">nav</span><span class="p">></span>
</code></pre></div>
<p>然后将 content 变量放在 'contaniner' 中</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"container content-body"</span><span class="p">></span>
%content%
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
</code></pre></div>
<h3 id="toc">替换样式和将 toc 移动到左侧</h3>
<p>替换表格样式</p>
<div class="highlight"><pre><span></span><code><span class="nx">$</span><span class="p">(</span><span class="s2">"table"</span><span class="p">).</span><span class="nx">addClass</span><span class="p">(</span><span class="s2">"table table-bordered table-striped table-hover"</span><span class="p">);</span>
</code></pre></div>
<p>然后将 <code>toc</code> 移动到右侧, 并使用 <code>affix</code> 效果</p>
<div class="highlight"><pre><span></span><code><span class="c1">// 生成左侧toc导航</span>
<span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">).</span><span class="nx">html</span><span class="p">().</span><span class="nx">trim</span><span class="p">()){</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">html</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".content-body"</span><span class="p">).</span><span class="nx">html</span><span class="p">();</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">toc_html</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'<div class="toc">'</span><span class="o">+</span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">).</span><span class="nx">html</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="s2">"</div>"</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">content</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">html</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">toc_html</span><span class="p">,</span><span class="w"> </span><span class="s2">""</span><span class="p">);</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">html</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'<div class="col-md-3">\n'</span><span class="o">+</span><span class="nx">toc_html</span><span class="o">+</span><span class="s1">'\n</div>\n'</span><span class="p">;</span>
<span class="w"> </span><span class="nx">html</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'<div class="col-md-9">\n'</span><span class="o">+</span><span class="nx">content</span><span class="o">+</span><span class="s1">'\n</div>\n'</span><span class="p">;</span>
<span class="w"> </span><span class="nx">html</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">html</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/blockquote/g</span><span class="p">,</span><span class="w"> </span><span class="s2">"pre"</span><span class="p">);</span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".content-body"</span><span class="p">).</span><span class="nx">html</span><span class="p">(</span><span class="nx">html</span><span class="p">);</span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">).</span><span class="nx">addClass</span><span class="p">(</span><span class="s2">"bs-sidebar"</span><span class="p">);</span>
<span class="w"> </span><span class="cm">/* $(".toc").attr("role", "complementary"); */</span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">).</span><span class="nx">attr</span><span class="p">(</span><span class="s2">"data-spy"</span><span class="p">,</span><span class="w"> </span><span class="s2">"affix"</span><span class="p">);</span>
<span class="w"> </span><span class="cm">/* $(".toc").attr("data-offset-top", "200") */</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">uls</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">).</span><span class="nx">find</span><span class="p">(</span><span class="s2">"ul"</span><span class="p">);</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kd">var</span><span class="w"> </span><span class="nx">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mf">0</span><span class="p">;</span><span class="w"> </span><span class="nx">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="nx">uls</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span><span class="w"> </span><span class="nx">i</span><span class="o">++</span><span class="p">){</span>
<span class="w"> </span><span class="nx">ul</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">uls</span><span class="p">[</span><span class="nx">i</span><span class="p">];</span>
<span class="w"> </span><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">parent</span><span class="p">(),</span><span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">));</span>
<span class="w"> </span><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">parent</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">".toc"</span><span class="p">));</span>
<span class="w"> </span><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">parent</span><span class="p">().</span><span class="nx">hasClass</span><span class="p">(</span><span class="s2">"toc"</span><span class="p">));</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">parent</span><span class="p">().</span><span class="nx">hasClass</span><span class="p">(</span><span class="s2">"toc"</span><span class="p">))</span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">addClass</span><span class="p">(</span><span class="s2">"nav bs-sidenav"</span><span class="p">);</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="nx">ul</span><span class="p">).</span><span class="nx">addClass</span><span class="p">(</span><span class="s2">"nav"</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>完整的<a href="http://wiki.linuxzen.com/static/js/wiki.js">wiki.js</a></p>
<h2 id="_5">自定义样式</h2>
<p>我同时还用 Vimwiki 做TODO管理, 原有的样式感觉挺不错的, 就拷贝了一份,
还有一点是从 bootcss.com 文档页面扒下来的,
可以在<a href="http://wiki.linuxzen.com/static/css/wiki.css">这里</a>看到.</p>
<h2 id="_6">部署</h2>
<p>我将我的vimwiki上传到了 <a href="https://bitbucket.org">bitbucket</a>, 然后添加了一个定义更新的脚本,
下面是脚本内容:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="nv">PWD</span><span class="o">=</span><span class="s2">"/home/yourusername/vimwiki"</span>
<span class="nb">cd</span><span class="w"> </span><span class="s2">"</span><span class="nv">$PWD</span><span class="s2">"</span>
git<span class="w"> </span>pull<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="s2">"Allready up-to-date."</span><span class="w"> </span><span class="m">2</span>><span class="p">&</span><span class="m">1</span><span class="w"> </span>><span class="w"> </span>/dev/null
<span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-ne<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="k">then</span>
<span class="w"> </span>vim<span class="w"> </span>+VimwikiIndex<span class="w"> </span>+VimwikiAll2HTML<span class="w"> </span>+qa
<span class="k">fi</span>
<span class="nb">cd</span><span class="w"> </span>-
</code></pre></div>
<p>上面脚本命名为 <code>update.sh</code> 保存在 <code>~/vimwiki</code> 下, 通过 <code>crontab</code> 进行定时调用</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>crontab<span class="w"> </span>-l
*/30<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>/bin/bash<span class="w"> </span>/home/yourusername/vimwiki/update.sh<span class="w"> </span><span class="p">&</span>><span class="w"> </span>/dev/null
</code></pre></div>
<h3 id="disqus">使用 Disqus 评论系统</h3>
<p>我在 <code>template/default.tpl</code> 添加了代码来使用 <code>Disqus</code> 评论系统, 这里不在详述</p>
<h3 id="wiki">已经上线的Wiki</h3>
<p>当然现在我已经部署了自己的wiki, 猛击<a href="http://wiki.linuxzen.com">这里</a>进入</p>
<h3 id="_7">接下来</h3>
<p>现在wiki还没有代码高亮, 我不想使用那个js的高亮, 想通过 <code>pygments</code> 来做代码高亮,
所以接下想通过 <code>pygments</code> 对代码进行高亮</p>
<h3 id="_8">参考</h3>
<ul>
<li><a href="http://www.berlinix.com/vim/vimwiki_with_bootstrap_jquery.php">用jQuery和Bootstrap美化VimWiki输出</a></li>
</ul>将Pelican版本更新到3.32013-12-24T00:00:00+08:002013-12-24T00:00:00+08:00coldtag:www.linuxzen.com,2013-12-24:/jiang-pelicanban-ben-geng-xin-dao-33.html<p>记录升级Pelican到3.3的过程和一些吐槽,</p><p>用Pelican有一段时间了, 由于重装了系统, 所以安装<code>Pelican</code>的最新版本,最新版本为 <code>3.3</code>, 顺便也升级模板和插件.</p>
<p><code>Pelican</code>特别喜欢更改配置文件, 而且错误提示非常烂, 完全不知道在说什么, 而且没有安装markdown包的话根本不提示, 仅仅就错误退出, 下面就这次升级越到的问题做个记录.</p>
<h2 id="makefile">Makefile</h2>
<p>Pelican升级后会有很多莫名奇妙的问题, 首先之前的<code>Makefile</code>不能使用, 所以需要重新生成一份:</p>
<div class="highlight"><pre><span></span><code>pelican-quickstart .
</code></pre></div>
<p>当然上面操作会更改配置文件, 我用git管理, 所以很方便的<code>checkout</code>, 如果你的不是, 先备份下配置文件吧.</p>
<h2 id="atom-feed">ATOM Feed</h2>
<p>然后<code>make html</code>的时候失败提示</p>
<div class="highlight"><pre><span></span><code><span class="n">File /path/to/project/output/feeds/all.atom.xml is to be overwritten!</span>
</code></pre></div>
<p>看了 <a href="https://github.com/getpelican/pelican">pelican</a>上的<code>issue</code> 原来是更改了<code>FEED_ATOM</code>的配置项, 使用了 <code>FEED_ALL_ATOM</code>配置项, 将<code>FEED_ATOM</code>配置项改为<code>FEED_ALL_ATOM</code>配置即可</p>
<p>但是改完之后页面的ATOM FEED链接没有指向正确的地址, 查看模板文件, 发现模板还在引用<code>FEED_ATOM</code>, 改成<code>FEED_ALL_ATOM</code>即可.</p>
<h2 id="markdown">Markdown</h2>
<p>如果莫名奇妙的错误, 没有错误信息, 也没有输出HTML, 那么可能就是<code>markdown</code>包没装, <code>Pelican</code>的错误提示真心无语.</p>
<div class="highlight"><pre><span></span><code>easy_install -U markdown
</code></pre></div>
<h3 id="markdown_1">Markdown 代码高亮</h3>
<p>Markdown 是通过指定 MD_EXTENSIONS 选项类配置代码高亮的之前配置这样就可以</p>
<div class="highlight"><pre><span></span><code><span class="n">MD_EXTENSIONS</span> <span class="o">=</span> <span class="p">([</span><span class="s1">'codehilite'</span><span class="p">,</span> <span class="s1">'extra'</span><span class="p">,</span> <span class="s1">'fenced_code'</span><span class="p">,</span> <span class="s1">'tables'</span><span class="p">,</span> <span class="s1">'sane_lists'</span><span class="p">])</span>
</code></pre></div>
<p>但是发现现在无法高亮代码, 查看了源码原来要手动指定高亮 css, 不然css会设置成
codhilite
<code>python
MD_EXTENSIONS = (['codehilite(css_class=highlight)', 'extra',
'fenced_code', 'tables', 'sane_lists'])</code></p>
<h2 id="_1">静态文件</h2>
<p><code>Pelican</code>去掉了<code>FILE\_TO\_COPY</code>项, 所以之前拷贝<code>robots.txt</code>之类的文件, 就会失效, 使用<code>STATIC_PATHS</code>即可</p>
<div class="highlight"><pre><span></span><code><span class="n">STATIC_PATHS</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="sa">u</span><span class="s2">"upload"</span><span class="p">,</span><span class="w"> </span><span class="s2">"extra/robots.txt"</span><span class="p">,</span>
<span class="w"> </span><span class="s2">"extra/404.html"</span><span class="p">,</span>
<span class="w"> </span><span class="p">]</span>
</code></pre></div>
<p>如果想指定静态文件位置, 可以使用<code>STATIC_SAVE_AS</code>, 会将静态文件存到另外一个目录,
而且对所有<code>STATIC_PATHS</code>项生效</p>
<div class="highlight"><pre><span></span><code>STATIC_SAVE_AS = "static/"
</code></pre></div>
<p>也可以使用 <code>EXTRA_PATH_METADATA</code>来为每一项指定路径</p>
<div class="highlight"><pre><span></span><code>EXTRA_PATH_METADATA = {
"extra/robots.txt":{"path":"robots.txt"},
"extra/404.html": {"path":"404.html"},
}
</code></pre></div>
<p>至此, <code>Pelican</code>升级完毕</p>PyQt + QML 快速开发GUI总结2013-11-06T15:25:00+08:002013-11-06T15:25:00+08:00coldtag:www.linuxzen.com,2013-11-06:/pyqt-qml-kuai-su-kai-fa-guizong-jie.html<p>最近结束一个使用PyQt+QML开发的项目, 在此对一些经验做出总结分享出来.
结合QML确实可以快速的构建出GUI程序, 但是相关资料太少, 特别是中文资料,
而且坑太多, 特别 …</p><p>最近结束一个使用PyQt+QML开发的项目, 在此对一些经验做出总结分享出来.
结合QML确实可以快速的构建出GUI程序, 但是相关资料太少, 特别是中文资料,
而且坑太多, 特别是和后端PyQt结合的时候有很多莫名奇妙的问题.
这篇文章会总结这些问题, 避免以后碰到无从下手.</p>
<p>PS:QML的一些基础问题不会在这里讨论, 本篇文章仅讨论一些经验性的问题, 本篇文章使用PyQt4</p>
<p><em>请留意文章中间的"注意"</em></p>
<div class="section" id="pyqt">
<h2>如何和PyQt交互</h2>
<p>QML和PyQt交互主要有三种方法: PyQt渲染数据, 信号传递, QML提供接口</p>
<div class="section" id="pyqtqml">
<h3>使用PyQt显示QML</h3>
<p>要想在Python里使用PyQt来调用QML显示, 需要用到 <tt class="docutils literal">PyQt4.QtDeclarative.QDeclarativeView</tt> 实例的 <tt class="docutils literal">setSource</tt> 将一个 <tt class="docutils literal">PyQt4.QtCore.QUrl</tt> 对象传递进去, 然后调用 <tt class="docutils literal">PyQt4.QtDeclarative.QDeclarativeView</tt> 对象的 <tt class="docutils literal">show</tt> 方法, 下面是一个例子:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4.QtDeclarative</span> <span class="kn">import</span> <span class="n">QDeclarativeView</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtCore</span> <span class="kn">import</span> <span class="n">QUrl</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">QDeclarativeView</span><span class="p">()</span>
<span class="n">view</span><span class="o">.</span><span class="n">setSource</span><span class="p">(</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"/path/to/demo.qml"</span><span class="p">))</span>
<span class="n">view</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
</div>
<div class="section" id="pyqtqml-1">
<h3>通过PyQt渲染数据到QML</h3>
<p>渲染QML变量有两种方法, 一种是QML中没有定义的变量, 一种是在QML中已经定义好的变量,
第一种是预定义变量, 第二种是设置变量</p>
<div class="section" id="section-1">
<h4>预定义变量</h4>
<p><tt class="docutils literal">PyQt4.QtDeclarative.QDeclarativeView</tt> 的 <tt class="docutils literal">rootContext</tt> 方法会返回一个QML上下文, 通过这个对象可以对QML进行一些变量的预定义</p>
<p><em>注意</em>, 预定义变量必须在 <tt class="docutils literal">setSource</tt> 调用之前进行, 还有如果是字符串类型如要使用 <tt class="docutils literal">PyQt4.QtCore.QString</tt> 进行转换,
数字可以不用转换, 如果是列表或字典需要使用 <tt class="docutils literal">PyQt4.QtCore.QVariant</tt> 进行转换, 下面是一个例子:</p>
<p>test.qml:</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">30</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="k">anchors.fill:</span><span class="nx">parent</span><span class="p">;</span>
<span class="k">text:</span> <span class="nx">textData</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>test.py:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4.QtDeclarative</span> <span class="kn">import</span> <span class="n">QDeclarativeView</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtCore</span> <span class="kn">import</span> <span class="n">QUrl</span><span class="p">,</span> <span class="n">QString</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">QDeclarativeView</span><span class="p">()</span>
<span class="n">rootContext</span> <span class="o">=</span> <span class="n">view</span><span class="o">.</span><span class="n">rootContext</span><span class="p">()</span>
<span class="n">rootContext</span><span class="o">.</span><span class="n">setContextProperty</span><span class="p">(</span><span class="s2">"textData"</span><span class="p">,</span> <span class="n">QString</span><span class="p">(</span><span class="s2">"hi"</span><span class="p">))</span>
<span class="n">view</span><span class="o">.</span><span class="n">setSource</span><span class="p">(</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"test.qml"</span><span class="p">))</span>
<span class="n">view</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>如果将 <tt class="docutils literal">rootContext</tt> 两行移到 <tt class="docutils literal">view.setSource</tt> 下面, QML里将找不到对 <tt class="docutils literal">textData</tt> 的引用</p>
</div>
<div class="section" id="section-2">
<h4>设置变量</h4>
<p>如果QML里已经定义好了变量, 而我们就可以在PyQt里对它进行更改, 对QML里的变量更改需要使用 <tt class="docutils literal">PyQt4.QtDeclarative.QDeclarativeView</tt> 对象的 <tt class="docutils literal">rootObject</tt> 方法返回的 <tt class="docutils literal">rootObject</tt> 对象, 调用 <tt class="docutils literal">rootObject</tt> 对象的 <tt class="docutils literal">setProperty</tt> 方法即可对QML里变量做出更改</p>
<p><em>注意</em>, 使用 <tt class="docutils literal">rootObject</tt> 对QML的更改必须在 <tt class="docutils literal">setSource</tt> 之后, 否则 <tt class="docutils literal">rootObject</tt> 方法将返回 <em>None</em>, 下面是一个例子</p>
<p>test.qml:</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span>
<span class="nx">property</span> <span class="nx">string</span> <span class="nx">textData</span><span class="p">;</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">30</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="k">anchors.fill:</span><span class="nx">parent</span><span class="p">;</span>
<span class="k">text:</span> <span class="nx">textData</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>test.py:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4.QtDeclarative</span> <span class="kn">import</span> <span class="n">QDeclarativeView</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtCore</span> <span class="kn">import</span> <span class="n">QUrl</span><span class="p">,</span> <span class="n">QString</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">QDeclarativeView</span><span class="p">()</span>
<span class="n">view</span><span class="o">.</span><span class="n">setSource</span><span class="p">(</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"test.qml"</span><span class="p">))</span>
<span class="n">rootObject</span> <span class="o">=</span> <span class="n">view</span><span class="o">.</span><span class="n">rootObject</span><span class="p">()</span>
<span class="n">rootObject</span><span class="o">.</span><span class="n">setProperty</span><span class="p">(</span><span class="s2">"textData"</span><span class="p">,</span> <span class="n">QString</span><span class="p">(</span><span class="s2">"hi"</span><span class="p">))</span>
<span class="n">view</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面例子我们在QML定义了 <tt class="docutils literal">textData</tt> 变量, 并在 <tt class="docutils literal">setSource</tt> 之后使用 <tt class="docutils literal">rootObject</tt> 的 <tt class="docutils literal">setProperty</tt> 对 <tt class="docutils literal">textData</tt> 变量进行了更改</p>
</div>
</div>
<div class="section" id="qml">
<h3>QML信号的传递</h3>
<p>QML有信号机制, 可以在QML之间使用JavaScript进行触发和接收, 当然也可以将信号传递给后端的PyQt,
我们在此不讨论QML内部的信号, 我们仅讨论QML信号传递到PyQt这部分, QML使用signal创建信号, 信号可以携带参数,
使用调用函数的方法可以触发信号, 下面是一个例子:</p>
<p>test.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span><span class="p">;</span>
<span class="nx">signal</span> <span class="nx">mclicked</span><span class="p">;</span> <span class="c1">// 定义信号</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="k">anchors.fill:</span><span class="nx">parent</span><span class="p">;</span>
<span class="k">text:</span> <span class="s2">"Click Me"</span>
<span class="p">}</span>
<span class="nx">MouseArea</span> <span class="p">{</span>
<span class="k">onClicked:</span> <span class="p">{</span>
<span class="nx">mclicked</span><span class="p">();</span> <span class="c1">// 触发信号</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>上面我们定义了一个 <tt class="docutils literal">mclicked</tt> 的信号, 并且在点击时会触发这个信号, PyQt 可以通过 <tt class="docutils literal">rootObject</tt> 获取这个信号, 并为这个信号绑定槽:</p>
<p>test.py</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4.QtDeclarative</span> <span class="kn">import</span> <span class="n">QDeclarativeView</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtCore</span> <span class="kn">import</span> <span class="n">QUrl</span><span class="p">,</span> <span class="n">QString</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">QDeclarativeView</span><span class="p">()</span>
<span class="n">view</span><span class="o">.</span><span class="n">setSource</span><span class="p">(</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"test.qml"</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">on_click</span><span class="p">():</span>
<span class="nb">print</span> <span class="s2">"hi"</span>
<span class="n">rootObject</span> <span class="o">=</span> <span class="n">view</span><span class="o">.</span><span class="n">rootObject</span><span class="p">()</span>
<span class="n">rootObject</span><span class="o">.</span><span class="n">mclicked</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">on_click</span><span class="p">)</span>
<span class="n">view</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面为QML的 <tt class="docutils literal">mclicked</tt> 信号绑定了一个函数, 当点击QML窗体时, 控制台就会输出 hi</p>
</div>
<div class="section" id="qml-1">
<h3>QML提供接口</h3>
<p>上面我们定义了信号, 如果响应信号仅仅在控制台输出没有意义, 我们可以通过QML在顶级元素定义JavaScript函数
向PyQt提供接口, 下面是例子:</p>
<p>test.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span><span class="p">;</span>
<span class="nx">signal</span> <span class="nx">mclicked</span><span class="p">;</span> <span class="c1">// 定义信号</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="kd">id: testText</span>
<span class="k">anchors.fill:</span><span class="nx">parent</span><span class="p">;</span>
<span class="k">text:</span> <span class="s2">"Click Me"</span>
<span class="p">}</span>
<span class="nx">MouseArea</span> <span class="p">{</span>
<span class="k">onClicked:</span> <span class="p">{</span>
<span class="nx">mclicked</span><span class="p">();</span> <span class="c1">// 触发信号</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">set_text</span><span class="p">(</span><span class="nx">text</span><span class="p">){</span>
<span class="nx">testText</span><span class="p">.</span><span class="nx">text</span> <span class="o">=</span> <span class="nx">text</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>上面我们在QML顶级元素定义了一个 <tt class="docutils literal">set_text</tt> 函数, 接下来我们就可以通过 <tt class="docutils literal">rootObject</tt> 进行调用</p>
<p>test.py</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4.QtDeclarative</span> <span class="kn">import</span> <span class="n">QDeclarativeView</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtCore</span> <span class="kn">import</span> <span class="n">QUrl</span><span class="p">,</span> <span class="n">QString</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">QDeclarativeView</span><span class="p">()</span>
<span class="n">view</span><span class="o">.</span><span class="n">setSource</span><span class="p">(</span><span class="n">QUrl</span><span class="p">(</span><span class="s2">"test.qml"</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">on_click</span><span class="p">():</span>
<span class="n">rootObject</span><span class="o">.</span><span class="n">set_text</span><span class="p">(</span><span class="s2">"Clicked"</span><span class="p">)</span>
<span class="n">rootObject</span> <span class="o">=</span> <span class="n">view</span><span class="o">.</span><span class="n">rootObject</span><span class="p">()</span>
<span class="n">rootObject</span><span class="o">.</span><span class="n">mclicked</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">on_click</span><span class="p">)</span>
<span class="n">view</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面我们用响应信号的槽, 通过 <tt class="docutils literal">rootObject</tt> 调用QML提供的函数接口, 当点击窗体时, 显示文字会从 <tt class="docutils literal">Click Me</tt> 变成 <tt class="docutils literal">Clicked</tt>,
当然这仅仅是个例子, 这种响应可以在QML里直接完成</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span><span class="p">;</span>
<span class="nx">signal</span> <span class="nx">mclicked</span><span class="p">;</span> <span class="c1">// 定义信号</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="kd">id: testText</span>
<span class="k">anchors.fill:</span><span class="nx">parent</span><span class="p">;</span>
<span class="k">text:</span> <span class="s2">"Click Me"</span>
<span class="p">}</span>
<span class="nx">MouseArea</span> <span class="p">{</span>
<span class="k">onClicked:</span> <span class="p">{</span>
<span class="nx">mclicked</span><span class="p">();</span> <span class="c1">// 触发信号</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">set_text</span><span class="p">(){</span>
<span class="nx">testText</span><span class="p">.</span><span class="nx">text</span> <span class="o">=</span> <span class="s2">"Clicked"</span>
<span class="p">}</span>
<span class="k">Component.onCompleted :</span> <span class="p">{</span>
<span class="nx">mclicked</span><span class="p">.</span><span class="nx">connect</span><span class="p">(</span><span class="nx">set_text</span><span class="p">);</span> <span class="c1">// 在加载完成后为mclicked信号绑定槽</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="section-3">
<h2>一些坑</h2>
<div class="section" id="qml-2">
<h3>QML全部变量内部状态更改</h3>
<p>QML定义使用 <tt class="docutils literal">property</tt> 定义的变量是全局的, 这些变量是无法对其内部状态进行更改的,
当然数字和字符串没有这方面问题, 但是数组和对象的更改就不行, 比如下面的更改是无效的</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span><span class="p">;</span>
<span class="nx">property</span> <span class="nx">variant</span> <span class="k">testData:</span> <span class="p">{</span><span class="s2">"a"</span><span class="o">:</span><span class="s2">"1"</span><span class="p">,</span> <span class="s2">"b"</span><span class="o">:</span> <span class="s2">"2"</span><span class="p">}</span>
<span class="nx">Repeater</span> <span class="p">{</span>
<span class="k">model:</span><span class="nx">testData</span><span class="p">;</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="k">text:</span> <span class="nx">modelData</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nx">MouseArea</span> <span class="p">{</span>
<span class="kd">id: testArea</span>
<span class="nx">anchors</span><span class="p">.</span><span class="nx">fill</span><span class="p">;</span>
<span class="k">onClicked:</span> <span class="p">{</span>
<span class="nx">testData</span><span class="p">.</span><span class="nx">a</span> <span class="o">=</span> <span class="s2">"0"</span>
<span class="nx">testData</span><span class="p">.</span><span class="nx">b</span> <span class="o">=</span> <span class="s2">"1"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>上面是无法更改testData的内部状态的, 解决办法是先将全局变量赋值给局部变量, 更改完毕后覆盖全局变量</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: test</span><span class="p">;</span>
<span class="nx">property</span> <span class="nx">variant</span> <span class="k">testData:</span> <span class="p">{</span><span class="s2">"a"</span><span class="o">:</span><span class="s2">"1"</span><span class="p">,</span> <span class="s2">"b"</span><span class="o">:</span> <span class="s2">"2"</span><span class="p">}</span>
<span class="nx">Repeater</span> <span class="p">{</span>
<span class="k">model:</span><span class="nx">testData</span><span class="p">;</span>
<span class="nx">Text</span> <span class="p">{</span>
<span class="k">text:</span> <span class="nx">modelData</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nx">MouseArea</span> <span class="p">{</span>
<span class="kd">id: testArea</span>
<span class="nx">anchors</span><span class="p">.</span><span class="nx">fill</span><span class="p">;</span>
<span class="k">onClicked:</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">tmp</span> <span class="o">=</span> <span class="nx">testData</span><span class="p">;</span> <span class="c1">// 赋值给局部变量</span>
<span class="nx">tmp</span><span class="p">.</span><span class="nx">a</span> <span class="o">=</span> <span class="s2">"0"</span>
<span class="nx">tmp</span><span class="p">.</span><span class="nx">b</span> <span class="o">=</span> <span class="s2">"1"</span>
<span class="nx">testData</span> <span class="o">=</span> <span class="nx">tmp</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>上面即可成功更改 <tt class="docutils literal">testData</tt> 的内部状态</p>
</div>
<div class="section" id="section-4">
<h3>空列表引发的段错误</h3>
<p>从PyQt向QML中渲染数据, 难免会渲染空列表, 但是这在某些平台(Windows)会引发段错误, 搞得人莫名其妙,
解决办法就是将空列表转换为 <em>None</em></p>
</div>
</div>
<div class="section" id="section-5">
<h2>一些技巧</h2>
<div class="section" id="js">
<h3>结合js</h3>
<p>给元素的属性赋值是可以使用js语句的, 当然复杂的语句可以写成函数,指定这个函数, 使用函数的返回值,
下面是一个隔行变色的例子:</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">400</span><span class="p">;</span>
<span class="nx">Repeater</span><span class="p">{</span>
<span class="k">model:</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">100</span><span class="p">;</span>
<span class="k">color:</span> <span class="nx">index</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span> <span class="o">?</span> <span class="s2">"blue"</span> <span class="o">:</span> <span class="s2">"black"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<div class="section" id="section-6">
<h3>自定义元素</h3>
<p>多处都使用的内容, 可以提取出来放在一个以大写开头的文件名的文件里, 然后同目录使用文件名就可以引用这个元素,
其他目录可以使用 <tt class="docutils literal">import</tt> 导入这个目录(相对路径导入), 就可以使用目录里的元素了,</p>
<p>比如我在 <tt class="docutils literal">Demo.qml</tt> 定义了一些内容, 同目录或导入这个目录的其他QML文件我就可以直接使用 <tt class="docutils literal">Demo</tt> 元素了</p>
</div>
<div class="section" id="section-7">
<h3>自定义元素属性</h3>
<p>为了更好的可重用, 定义一些属性是必须的 可以用通过 <tt class="docutils literal">property alias source:target</tt> 定义属性别名, 当别人引用
这个元素, 并在元素内定义 <tt class="docutils literal">source</tt> 属性, 就会自动映射到 <tt class="docutils literal">target</tt> 上, 下面是一个例子</p>
<p>Demo.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: demo</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">50</span><span class="p">;</span>
<span class="nx">property</span> <span class="nx">alias</span> <span class="k">text:</span> <span class="nx">demoText</span><span class="p">.</span><span class="nx">text</span><span class="p">;</span>
<span class="nx">Text</span><span class="p">{</span>
<span class="kd">id: demoText</span><span class="p">;</span>
<span class="k">text:</span> <span class="s2">"Demo"</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>test.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Demo</span> <span class="p">{</span>
<span class="k">text:</span> <span class="s2">"Nice Demo"</span>
<span class="p">}</span>
</pre></div>
<p>使用qmlviwer 查看 test.qml, 会显示 "Nice Demo" 文字</p>
<p>当然也可以直接在自定义元素使用 <tt class="docutils literal">property</tt> 定义变量, 在使用该元素时定义该属性:</p>
<p>Demo.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Rectangle</span> <span class="p">{</span>
<span class="kd">id: demo</span>
<span class="k">width:</span> <span class="mi">100</span><span class="p">;</span> <span class="k">height:</span> <span class="mi">50</span><span class="p">;</span>
<span class="nx">property</span> <span class="nx">string</span> <span class="k">textData:</span><span class="s2">"Demo"</span><span class="p">;</span>
<span class="nx">Text</span><span class="p">{</span>
<span class="kd">id: demoText</span><span class="p">;</span>
<span class="k">text:</span> <span class="nx">textData</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>test.qml</p>
<div class="highlight"><pre><span></span><span class="kr">import</span> <span class="nx">Qt</span> <span class="mf">4.7</span>
<span class="nx">Demo</span> <span class="p">{</span>
<span class="k">textData:</span> <span class="s2">"Nice Demo"</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
推荐几款最近发现非常酷的Vim插件2013-10-18T13:14:00+08:002013-10-18T13:14:00+08:00coldtag:www.linuxzen.com,2013-10-18:/tui-jian-ji-kuan-zui-jin-fa-xian-fei-chang-ku-de-vimcha-jian.html<p>最近看一个github上的Vim配置, 发现了几款非常酷而且非常有用的Vim插件:</p>
<ul class="simple">
<li>delimitMate
用于补全括号和引号</li>
<li>vim-surround
用于快速切换括号/引号或者标签</li>
<li>GitGutter
实时 …</li></ul><p>最近看一个github上的Vim配置, 发现了几款非常酷而且非常有用的Vim插件:</p>
<ul class="simple">
<li>delimitMate
用于补全括号和引号</li>
<li>vim-surround
用于快速切换括号/引号或者标签</li>
<li>GitGutter
实时显示git更改</li>
<li>Gitv
查看Git详细提交日志(类似gitk)</li>
<li>vim-commentary
Vim批量注释工具, 可以注释多行和去除多行注释</li>
<li>indentLine
更加美观的显示缩进对齐线</li>
</ul>
<p>先放上录屏:</p>
<div style="width:800; height: 500">
<script type="text/javascript" src="http://asciinema.org/a/5981.js" id="asciicast-5981" async></script>
</div><div class="section" id="section-1">
<h2>安装</h2>
<p>上面插件可以通过 Vundle 来安装 (了解Vundle猛击 <a class="reference external" href="http://www.linuxzen.com/vimpei-zhi-xi-lie-cha-jian-guan-li.html">这里</a>), 下面是 <tt class="docutils literal">.vimrc</tt> 的配置</p>
<div class="highlight"><pre><span></span>Bundle <span class="s2">"Yggdroot/indentLine"</span>
Bundle <span class="s2">"airblade/vim-gitgutter"</span>
Bundle <span class="s2">"gregsexton/gitv"</span>
Bundle <span class="s2">"tpope/vim-commentary"</span>
Bundle <span class="s2">"tpope/vim-surround"</span>
Bundle <span class="s2">"Raimondi/delimitMate"</span>
</pre></div>
<p>然后重新打开 Vim, 执行 <tt class="docutils literal">:BundleIntall</tt> 等待安装完成</p>
</div>
<div class="section" id="section-2">
<h2>配置使用</h2>
<p>delimitMate和GitGutter安装完成不用任何配置即可使用, 下面我们先介绍 <tt class="docutils literal"><span class="pre">vim-surround</span></tt> 插件的使用</p>
<div class="section" id="vim-surround">
<h3>vim-surround</h3>
<p>这个插件可以快速的为字符串包围/改变或去除引号/括号或者HTML标签</p>
<div class="section" id="section-3">
<h4>为单个单词包围</h4>
<p>在命令模式下, 使用 <tt class="docutils literal">ysiw</tt> + <tt class="docutils literal"><span class="pre">'/"/(/[/{</span></tt> 就可以为光标下的一个单词包围上 <tt class="docutils literal"><span class="pre">'/"/(/[/{</span></tt></p>
<p>比如 <tt class="docutils literal">ysiw'</tt> 为光标下的单词包围上单引号, <tt class="docutils literal">ysiw"</tt> 为光标下单词包围上双引号, 依此类推.</p>
<p>vim-surround 同时还支持包围html标签, 将光标放到某单词试试下面指令</p>
<div class="highlight"><pre><span></span>ysiw<span class="p"><</span><span class="k">p</span><span class="p">></span>
ysiw<span class="p"><</span><span class="k">p</span> class<span class="p">=</span><span class="s2">"meta"</span><span class="p">></span>
</pre></div>
</div>
<div class="section" id="section-4">
<h4>包围一行</h4>
<p><tt class="docutils literal">yssb</tt> 可以快速为一行包围圆括号, <tt class="docutils literal">yss</tt> + <tt class="docutils literal"><span class="pre">'/"/(/[/{</span></tt> 可以为正行快速包围相应的引号/括号</p>
<p>比如 <tt class="docutils literal">yss"</tt> 为一行包围双引号</p>
</div>
<div class="section" id="section-5">
<h4>更改包围</h4>
<p><tt class="docutils literal">cs</tt> 指令可以更改包围, 比如 <tt class="docutils literal">cs'"</tt> 是将单引号变成双引号, <tt class="docutils literal">cs"(</tt> 是将双引号变成圆括号</p>
<p>vim-surround支持将括号或者引号变更为html标签, 试试下面命令</p>
<div class="highlight"><pre><span></span><span class="k">cs</span>'<span class="p"><</span><span class="k">p</span><span class="p">></span>
</pre></div>
<p>上面命令将单引号换成 <tt class="docutils literal"><p></tt> 标签</p>
</div>
<div class="section" id="section-6">
<h4>去除包围</h4>
<p><tt class="docutils literal">ds</tt> 指令可以取出包围, 后面需跟包围的内容, <tt class="docutils literal">ds"</tt> 是去除双引号包围, ``</p>
</div>
</div>
<div class="section" id="indentline">
<h3>indentLine</h3>
<p>这个插件安装成功后就会显示缩进对齐线, 我们仅仅在 <tt class="docutils literal">.vimrc</tt> 里加一行来切换是否显示</p>
<div class="highlight"><pre><span></span>map <span class="p"><</span>leader<span class="p">></span><span class="k">il</span> :IndentLinesToggle<span class="p"><</span>CR<span class="p">></span>
</pre></div>
<p>这样我们就可以通过 <tt class="docutils literal"><leader> il</tt> (我的leader映射的,)来切换是否显示对齐线</p>
</div>
<div class="section" id="gitv">
<h3>Gitv</h3>
<p>Gitv 实现了可以用Vim来查看Git的详细提交信息, 只需要打开Vim 执行 <tt class="docutils literal">:Gitv</tt></p>
</div>
<div class="section" id="vim-commentary">
<h3>vim-commentary</h3>
<p>这个插件可以快速注释与反注释多行内容, 但是它的注释符使用的是 <tt class="docutils literal">commentstring</tt>, 默认是 <tt class="docutils literal">/* %s */</tt>, 但这个值满足不了Python 和 Shell这样的语言,
在 <tt class="docutils literal">.vimrc</tt> 添加如下内容</p>
<div class="highlight"><pre><span></span>autocmd <span class="nb">FileType</span> python<span class="p">,</span><span class="k">shell</span> <span class="k">set</span> <span class="nb">commentstring</span><span class="p">=</span>#\ %s <span class="c">" 设置Python注释字符</span>
autocmd <span class="nb">FileType</span> mako <span class="k">set</span> <span class="nb">cms</span><span class="p">=</span>##\ %s
</pre></div>
<p><tt class="docutils literal">Visual</tt> 模式下 <tt class="docutils literal">gc</tt> 命令可以注释选中的行</p>
<p>普通模式下 <tt class="docutils literal">gcc</tt> 指令可以快速注释一行</p>
<p><tt class="docutils literal">gcu</tt> 可以撤销注释</p>
</div>
</div>
<div class="section" id="section-7">
<h2>最后</h2>
<p>有什么没介绍到的大家可以看看帮助, 大家也可以围观这个强大Vim配置: <a class="reference external" href="https://github.com/liangxianzhe/dotvim">https://github.com/liangxianzhe/dotvim</a></p>
</div>
Vim 相对行号2013-10-09T14:50:00+08:002013-10-09T14:50:00+08:00coldtag:www.linuxzen.com,2013-10-09:/vim-xiang-dui-xing-hao.html<p class="first last">大家在用Vim时是否会遇到想复制多行或者想快速向下/向上移动多行时而不知道行数的情况, 相对行号将帮助你轻易的完成类似的任务</p>
<p>在使用用Vim时时常会遇到像复制多行或者想快速向下/向上移动多行时而不知道行数的情况, 今天发现一个Vim对此有帮助的特性: <strong>相对行号</strong>.
相对行号将会在每行前显示相对于光标所在行的行号, 相对行号你给出某些垂直移动命令时需要的计数值</p>
<p>在 <tt class="docutils literal">Vim 7.4</tt> 之前的版本, 启用相对行号当前行号将变为0, 无法显示正常的行号, 在 <tt class="docutils literal">Vim 7.4</tt> 中则可以正常的显示当前行号的同时来显示相对行号, 下面有两张图可以对比</p>
<p>Vim 7.3</p>
<img alt="Vim 7.3" src="/static/upload/vim_rn_pre_74.png" />
<p>Vim 7.4</p>
<img alt="Vim 7.3" src="/static/upload/vim_rn_74.png" />
<p>从上面的图片看来, 启用相对行号后可以很直观的看到当前行距离上面/下面某行的行数</p>
<p>可以使用以下指令启用这个特性</p>
<div class="highlight"><pre><span></span>set relativenumber
</pre></div>
<p>启用相对行号后再也不用为了复制多行费力气的去数了, so easy!</p>
PyQt4 信号和槽详解2013-09-22T10:10:00+08:002013-09-22T10:10:00+08:00coldtag:www.linuxzen.com,2013-09-22:/pyqt4-xin-hao-he-cao-xiang-jie.html<p class="first last">最近在开发一个基于 <tt class="docutils literal">PyQt4</tt> 的的 GUI程序, 使用过程中一些对信号和槽的理解分享给大家</p>
<p>Python 可以开发GUI有很多库可以选择, 之前使用过 <tt class="docutils literal">wxPython</tt>, 已经很长时间没用过, 基本都忘光了, 由于单位也使用 <tt class="docutils literal">PyQt</tt> , 之前熟悉项目时看过一段时间, 但是由于没有实际项目经验, 所以 <tt class="docutils literal">PyQt</tt> 很生疏, 最近正好给家里写个小的财务软件练练手. 在用的过程中对 <tt class="docutils literal">PyQt</tt> 熟悉不少, 之前觉得 <tt class="docutils literal">PyQt</tt> 最难掌握的就是 <tt class="docutils literal">信号</tt> 和 <tt class="docutils literal">槽</tt> , 所以现在先来讨论这个</p>
<div class="section" id="section-1">
<h2>概览</h2>
<p>信号和槽可以说是 <tt class="docutils literal">Qt</tt> 的精髓所在, 有些类似 Javascript 的事件响应机制, 所以咱先以 <tt class="docutils literal">JQuery</tt> 来做个示例, 在 <tt class="docutils literal">JQuery</tt> 中我们获取一个元素, 并且绑定一个事件, 传递一个匿名函数来响应该事件</p>
<div class="highlight"><pre><span></span><span class="cm"><!-- 需导入 jquery 库 --></span>
<span class="p"><</span><span class="nt">button</span> <span class="na">id</span><span class="o">=</span><span class="s">"test"</span><span class="p">></span>点我<span class="p"></</span><span class="nt">button</span><span class="p">></span>
<span class="p"><</span><span class="nt">script</span> <span class="na">type</span><span class="o">=</span><span class="s">"text/javascript"</span><span class="p">></span>
<span class="w"> </span><span class="nx">$</span><span class="p">(</span><span class="s2">"#test"</span><span class="p">).</span><span class="nx">click</span><span class="p">(</span><span class="kd">function</span><span class="p">(){</span>
<span class="w"> </span><span class="nx">alert</span><span class="p">(</span><span class="s2">"点疼我了!!!"</span><span class="p">);</span>
<span class="w"> </span><span class="p">)};</span>
<span class="p"></</span><span class="nt">script</span><span class="p">></span>
</pre></div>
<p>以上代码, 当点击按钮 <tt class="docutils literal">点我</tt> 时会弹出一个对话框, 这就是响应了点击事件.</p>
<p>信号和槽有着类似的机制, <tt class="docutils literal">QObject.connect</tt> 可以连接一个 <tt class="docutils literal">QObject</tt> 的信号到另一个 <tt class="docutils literal">QObject</tt> 的槽, <tt class="docutils literal">PyQt</tt> 同时可以将信号连接到一个 <tt class="docutils literal">callback</tt></p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">():</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"ok"</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">(</span><span class="sa">u</span><span class="s2">"点我"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面例子将一个 button 对象的 <tt class="docutils literal">clicked()</tt> 信号连接到 <tt class="docutils literal">showMsg</tt> 函数. 也就是说 <tt class="docutils literal">ShowMsg</tt> 响应了一个按钮的点击事件.</p>
<p>PyQt 有一种类似 <tt class="docutils literal">JQuery</tt> 响应事件的方式, 来连接信号和槽</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">():</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"ok"</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">(</span><span class="sa">u</span><span class="s2">"点我"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">btn</span><span class="o">.</span><span class="n">clicked</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">showMsg</span><span class="p">)</span> <span class="c1"># XXX 此处与上面例子不同</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
</div>
<div class="section" id="section-2">
<h2>信号</h2>
<p>要想了解信号的本质, 我们需要了解信号的创建和 <tt class="docutils literal">QObject.emit</tt> 方法, <tt class="docutils literal">emit</tt> 方法用来发射信号</p>
<div class="section" id="section-3">
<h3>定义信号</h3>
<p><tt class="docutils literal">PyQt4.QtCore.pyqtSignal</tt> 函数可以为 <tt class="docutils literal">QObject</tt> 创建一个信号</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">)</span>
<span class="n">myclicked</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">pyqtSignal</span><span class="p">()</span>
</pre></div>
<p>上面例子为 <tt class="docutils literal">MyButton</tt> 创建了一个 <tt class="docutils literal">myclicked()</tt> 的信号</p>
</div>
<div class="section" id="section-4">
<h3>带参数的信号</h3>
<p>信号可以携带参数, 并在发射信号时携带传递给槽</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">)</span>
<span class="n">myclicked</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">pyqtSignal</span><span class="p">(</span><span class="nb">int</span><span class="p">)</span>
</pre></div>
<p>上面例子定义了 <tt class="docutils literal">myclicked(int)</tt> 信号, 可以携带一个, 发射时信号时可以携带一个整数</p>
</div>
<div class="section" id="section-5">
<h3>发射信号</h3>
<p>为了发射我们自定义的信号, 我们对 <tt class="docutils literal">QPushButton</tt> 进行一下封装, 自动绑定 <tt class="docutils literal">clicked()</tt> 信号, 并发射自定义的信号</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">):</span>
<span class="n">myclicked</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">pyqtSignal</span><span class="p">()</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">myclicked</span><span class="o">.</span><span class="n">emit</span><span class="p">)</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">():</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"ok"</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="sa">u</span><span class="s2">"点我"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked()"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面我们封装了 <tt class="docutils literal">QPushButton</tt> 让他在收到点击信号时同时发送 <tt class="docutils literal">myclicked()</tt> 信号.</p>
<p>我们也可以不定义信号, 直接发送信号, 上面的例子也可以这么写</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">emitClicked</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">emitClicked</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">emit</span><span class="p">(</span><span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked()"</span><span class="p">))</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">():</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"ok"</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="sa">u</span><span class="s2">"点我"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked()"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面例子我们没有定义信号, 仅仅是在响应 <tt class="docutils literal">clicked()</tt> 信号的函数内直接发送 <tt class="docutils literal">myclicked()</tt> 信号</p>
</div>
<div class="section" id="section-6">
<h3>发射带参数的信号</h3>
<p>有时我们展示了一个列表, 并想提供查看某项列表的详细内容, 我们会在列表项的末端加一个查看按钮, 这时我们如何在按按钮的时候得知这是那一项呢? 这时就需要带参数的信号, 信号是可以带参数的, 参数会在信号发送时携带, 并传递给接收此信号的槽</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">):</span>
<span class="n">myclicked</span> <span class="o">=</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">pyqtSignal</span><span class="p">(</span><span class="nb">int</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">_id</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_id</span> <span class="o">=</span> <span class="n">_id</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">emitMyclicked</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">emitMyclicked</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">myclicked</span><span class="o">.</span><span class="n">emit</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_id</span><span class="p">)</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="n">w</span><span class="o">.</span><span class="n">resize</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">(</span><span class="n">_id</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看 </span><span class="si">%d</span><span class="s2">"</span> <span class="o">%</span> <span class="n">_id</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看1"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked(int)"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">btn2</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看2"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">btn2</span><span class="o">.</span><span class="n">move</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">30</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn2</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked(int)"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面例子可以看出, <tt class="docutils literal">QObject.emit</tt> 发送带参数的信号时要携带参数. 当然上面例子也可以用下面方式来写</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MyButton</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">_id</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_id</span> <span class="o">=</span> <span class="n">_id</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">emitClicked</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">emitClicked</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">emit</span><span class="p">(</span><span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked(int)"</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">_id</span><span class="p">)</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">()</span>
<span class="n">w</span><span class="o">.</span><span class="n">resize</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">showMsg</span><span class="p">(</span><span class="n">_id</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看 </span><span class="si">%d</span><span class="s2">"</span> <span class="o">%</span> <span class="n">_id</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看1"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked(int)"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">btn2</span> <span class="o">=</span> <span class="n">MyButton</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"查看2"</span><span class="p">,</span> <span class="n">w</span><span class="p">)</span>
<span class="n">btn2</span><span class="o">.</span><span class="n">move</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">30</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn2</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"myclicked(int)"</span><span class="p">),</span> <span class="n">showMsg</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="section" id="section-7">
<h2>槽</h2>
<p>我一开始学习 PyQt 的信号和槽的时候看到那个 <a class="reference external" href="http://jimmykuu.sinaapp.com/static/PyQt4_Tutorial/html/events_and_signals.html#id1">滑块的例子</a> 一直搞的我很迷糊, 不知所以, 也没学会怎么用.
这里咱就自己创建一个槽, 就能了解什么是槽和槽该怎么用.</p>
<p>上面我们一直使用 <tt class="docutils literal">函数</tt> (callback)作为槽, 下面我们来介绍使用 <cite>真正</cite> 的槽</p>
<div class="section" id="section-8">
<h3>创建槽</h3>
<p><tt class="docutils literal">QtCore.pyqtSlot</tt> 函数返回一个 <tt class="docutils literal">装饰器</tt> 用于装饰 <tt class="docutils literal">QObject</tt> 的方法, 使之成为一个槽(我开始一直以为 <tt class="docutils literal">QObject</tt> 的一个方法就是一个槽 囧rz), 下面例子我们创建一个槽</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">PyQt4</span> <span class="kn">import</span> <span class="n">QtGui</span><span class="p">,</span> <span class="n">QtCore</span>
<span class="k">class</span> <span class="nc">MainWidget</span><span class="p">(</span><span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QWidget</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QPushButton</span><span class="p">(</span><span class="sa">u</span><span class="s2">"点我"</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">btn</span><span class="p">,</span> <span class="n">QtCore</span><span class="o">.</span><span class="n">SIGNAL</span><span class="p">(</span><span class="s2">"clicked()"</span><span class="p">),</span> <span class="bp">self</span><span class="p">,</span>
<span class="n">QtCore</span><span class="o">.</span><span class="n">SLOT</span><span class="p">(</span><span class="s2">"onClicked()"</span><span class="p">))</span>
<span class="nd">@QtCore</span><span class="o">.</span><span class="n">pyqtSlot</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">onClicked</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">QtGui</span><span class="o">.</span><span class="n">QMessageBox</span><span class="o">.</span><span class="n">information</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"信息"</span><span class="p">,</span> <span class="sa">u</span><span class="s2">"由槽弹出"</span><span class="p">)</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QtGui</span><span class="o">.</span><span class="n">QApplication</span><span class="p">([])</span>
<span class="n">m</span> <span class="o">=</span> <span class="n">MainWidget</span><span class="p">()</span>
<span class="n">m</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">exec_</span><span class="p">()</span>
</pre></div>
<p>上面例子我们为 <tt class="docutils literal">MainWidget</tt> 创建了一个槽, 并将 <tt class="docutils literal">btn</tt> 的 <tt class="docutils literal">clicked()</tt> 信号连接到这个槽</p>
<p>本文所有例子都是经过测试可以运行的, 所以大家如果对信号和槽还是有点迷糊不妨将例子中的代码敲一下并运行, 建议改改例子, 解答自己的疑惑</p>
</div>
</div>
Linux 下 Python 实现按任意键退出2013-08-20T09:54:00+08:002013-08-20T09:54:00+08:00coldtag:www.linuxzen.com,2013-08-20:/linux-xia-python-shi-xian-an-ren-yi-jian-tui-chu.html<p>初学Python时在总想实现一个按任意键继续/退出的程序(受.bat毒害), 奈何一直写不出来, 最近学习Unix C时发现可以通过<code>termios.h</code>库来实现, 尝试一下发现Python也有这个库, 所以终于写出一个这样的程序 …</p><p>初学Python时在总想实现一个按任意键继续/退出的程序(受.bat毒害), 奈何一直写不出来, 最近学习Unix C时发现可以通过<code>termios.h</code>库来实现, 尝试一下发现Python也有这个库, 所以终于写出一个这样的程序. 下面是代码:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">termios</span>
<span class="k">def</span> <span class="nf">press_any_key_exit</span><span class="p">(</span><span class="n">msg</span><span class="p">):</span>
<span class="c1"># 获取标准输入的描述符</span>
<span class="n">fd</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdin</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="c1"># 获取标准输入(终端)的设置</span>
<span class="n">old_ttyinfo</span> <span class="o">=</span> <span class="n">termios</span><span class="o">.</span><span class="n">tcgetattr</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="c1"># 配置终端</span>
<span class="n">new_ttyinfo</span> <span class="o">=</span> <span class="n">old_ttyinfo</span><span class="p">[:]</span>
<span class="c1"># 使用非规范模式(索引3是c_lflag 也就是本地模式)</span>
<span class="n">new_ttyinfo</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">&=</span> <span class="o">~</span><span class="n">termios</span><span class="o">.</span><span class="n">ICANON</span>
<span class="c1"># 关闭回显(输入不会被显示)</span>
<span class="n">new_ttyinfo</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">&=</span> <span class="o">~</span><span class="n">termios</span><span class="o">.</span><span class="n">ECHO</span>
<span class="c1"># 输出信息</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="c1"># 使设置生效</span>
<span class="n">termios</span><span class="o">.</span><span class="n">tcsetattr</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">termios</span><span class="o">.</span><span class="n">TCSANOW</span><span class="p">,</span> <span class="n">new_ttyinfo</span><span class="p">)</span>
<span class="c1"># 从终端读取</span>
<span class="n">os</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
<span class="c1"># 还原终端设置</span>
<span class="n">termios</span><span class="o">.</span><span class="n">tcsetattr</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">termios</span><span class="o">.</span><span class="n">TCSANOW</span><span class="p">,</span> <span class="n">old_ttyinfo</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">press_any_key_exit</span><span class="p">(</span><span class="s2">"按任意键继续..."</span><span class="p">)</span>
<span class="n">press_any_key_exit</span><span class="p">(</span><span class="s2">"按任意键退出..."</span><span class="p">)</span>
</code></pre></div>
<p>其他关于<code>termios</code>的信息可以参考Linux手册:</p>
<div class="highlight"><pre><span></span><code>man<span class="w"> </span><span class="m">3</span><span class="w"> </span>termios
</code></pre></div>
<p>另补充一下*nix终端的三种模式(摘自<Unix-Linux编程实践教程>)</p>
<h3 id="_1">规范模式</h3>
<p>规范模式, 也被成为cooked模式, 是用户常见的模式.驱动程序输入的字符保存在缓冲区, 并且仅在接收到回车键时才将这些缓冲的字符发送到程序.缓冲数据使驱动程序可以实现最基本的编辑功能, 被指派这些功能的特定键在驱动程序里设置, 可以通过命令stty或系统调用tcsetattr来修改</p>
<h3 id="_2">非规范模式</h3>
<p>当缓冲和编辑功能被关闭时, 连接被成为非规范模式.终端处理器仍旧进行特定的字符处理, 例如处理Ctrl-C及换行符之间的转换, 但是编辑键将没有意义, 因此相应的输入被视为常规的数据输入
程序需要自己实现编辑功能</p>
<h3 id="raw">raw模式</h3>
<p>当所有处理都被关闭后, 驱动程序将输入直接传递给程序, 连接被成为raw模式.</p>Vim 代码补全和检查: YouCompleteMe & syntastic2013-08-14T00:00:00+08:002013-08-14T00:00:00+08:00coldtag:www.linuxzen.com,2013-08-14:/vim-dai-ma-bu-quan-he-jian-cha-youcompleteme-syntastic.html<p>Vim 7.4 发布, 最近升级了Vim, 并安装了YouCompleteMe和Syntastic插件, 这里记录下过程</p>
<h2 id="vim">升级Vim</h2>
<p>YouCompleteMe 需要Vim 7.3.584+的支持, 并且开启 +python , 可以通过<code>:version</code>查看
升级 …</p><p>Vim 7.4 发布, 最近升级了Vim, 并安装了YouCompleteMe和Syntastic插件, 这里记录下过程</p>
<h2 id="vim">升级Vim</h2>
<p>YouCompleteMe 需要Vim 7.3.584+的支持, 并且开启 +python , 可以通过<code>:version</code>查看
升级Vim需要先卸载原有的Vim</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>remove<span class="w"> </span>vim<span class="w"> </span>vim-tiny<span class="w"> </span>vim-common<span class="w"> </span>vim-runtime<span class="w"> </span>gvim<span class="w"> </span>vim-gui-common
</code></pre></div>
<p>并安装以下依赖</p>
<div class="highlight"><pre><span></span><code>sudo apt-get install libncurses5-dev libgnome2-dev libgnomeui-dev \
libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
libcairo2-dev libx11-dev libxpm-dev libxt-dev \
python-dev ruby-dev mercurial checkinstall
</code></pre></div>
<p>下载最新的Vim源码, 这里从代码仓库获取(需要<code>hg</code>没有自行安装)</p>
<div class="highlight"><pre><span></span><code>hg<span class="w"> </span>clone<span class="w"> </span>https://vim.googlecode.com/hg/<span class="w"> </span>vim
</code></pre></div>
<p>然后进入目录编译安装Vim</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>vim
./configure<span class="w"> </span>--with-features<span class="o">=</span>huge<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-rubyinterp<span class="o">=</span>yes<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-pythoninterp<span class="o">=</span>yes<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-python3interp<span class="o">=</span>yes<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-perlinterp<span class="o">=</span>yes<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-luainterp<span class="w"> </span><span class="o">=</span><span class="w"> </span>yes<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--enable-gui<span class="o">=</span>gtk2<span class="w"> </span>--enable-cscope<span class="w"> </span>--prefix<span class="o">=</span>/usr
make<span class="w"> </span><span class="nv">VIMRUNTIMEDIR</span><span class="o">=</span>/usr/share/vim/vim74
sudo<span class="w"> </span>checkinstall
</code></pre></div>
<h2 id="llvm">安装llvm</h2>
<p>如果想是想C系语言的补全, 需要libclang 3.2以上的版本, Ubuntu 12.10 自带的是3.0, 所以先安装LLVM, 可以下载二进制文件/编译安装</p>
<h4 id="_1">下载二进制</h4>
<p>到<a href="http://llvm.org/releases/download.html#3.3">llvm.org</a>上下载相应的版本解压到~/ycm_temp</p>
<h4 id="_2">编译</h4>
<p>下载<a href="http://llvm.org/releases/3.3/cfe-3.3.src.tar.gz">clang</a>和<a href="http://llvm.org/releases/3.3/llvm-3.3.src.tar.gz">llvm</a>, 解压llvm</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>~/ycm_temp
<span class="nb">cd</span><span class="w"> </span>~/ycm_temp
tar<span class="w"> </span>-zxvf<span class="w"> </span>llvm-3.3.src.tar.gz<span class="w"> </span>-C<span class="w"> </span>llvm.src
</code></pre></div>
<p>解压 clang到llvm.src/tools</p>
<div class="highlight"><pre><span></span><code>tar<span class="w"> </span>-zxvf<span class="w"> </span>cfe-3.3.src.tar.gz<span class="w"> </span>-C<span class="w"> </span>~/ycm_temp/llvm.src/tools/
mv<span class="w"> </span>~/ycm_temp/llvm.src/tools/cfe-3.3.src<span class="w"> </span>~/ycm_temp/llvm/tools/clang
</code></pre></div>
<p>编译llvm会自动编译clang</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>~/ycm_temp
mkdir<span class="w"> </span>llvm_build
<span class="nb">cd</span><span class="w"> </span>llvm_build
cmake<span class="w"> </span>../llvm.src/CMakeList.txt<span class="w"> </span>../llvm.src
make
</code></pre></div>
<h2 id="youcompleteme">安装YouCompleteMe</h2>
<p>使用Vundle安装YouComplete(猛击<a href="/vimpei-zhi-xi-lie-cha-jian-guan-li.html">这里</a>了解Vundle)</p>
<h3 id="ycm_core">编译ycm_core</h3>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>~/ycm_build
<span class="nb">cd</span><span class="w"> </span>~/ycm_build
cmake<span class="w"> </span>-G<span class="w"> </span><span class="s2">"Unix Makefiles"</span><span class="w"> </span>~/.vim/bundle/YouCompleteMe/cpp<span class="w"> </span>-DEXTERNAL_LIBCLANG_PATH<span class="o">=</span>~/ycm_temp/llvm.src/lib/libclang.so<span class="w"> </span>
make<span class="w"> </span>ycm_core
</code></pre></div>
<p>下载llvm二进制的可以参考</p>
<div class="highlight"><pre><span></span><code>cmake<span class="w"> </span>-G<span class="w"> </span><span class="s2">"Unix Makefiles"</span><span class="w"> </span>~/.vim/bundle/YouCompleteMe/cpp<span class="w"> </span>-DEXTERNAL_LIBCLANG_PATH<span class="o">=</span>~/ycm_temp/llvm_root_path/lib/libclang.so
</code></pre></div>
<h3 id="_3">配置</h3>
<p>YouCompleteMe 需要一个配置文件来补全, 可以参考<a href="https://github.com/Valloric/YouCompleteMe/blob/master/cpp/ycm/.ycm_extra_conf.py">官方配置文件</a>, 可以将配置文件至于项目根目录或者上级目录, YouCompleteMe 会自动检测加载, 也可以通过指定<code>g:ycm_global_ycm_extra_conf</code>指定一个全局的配置文件</p>
<p>YouCompleteMe 每次加载配置文件会有一个提示, 很烦人, 可以通过将<code>g:ycm_confirm_extra_conf</code>置为0关闭提示</p>
<h3 id="python">补全Python</h3>
<p>YouCompleteMe 通过 <code>jedi</code>插件来补全Python, 可以通过<code>Vundle</code>安装此插件,在.vimrc中添加</p>
<div class="highlight"><pre><span></span><code>Bundle "davidhalter/jedi"
</code></pre></div>
<h2 id="_4">代码检查</h2>
<p><code>syntastic</code>是一个代码检查的插件, 通过<code>Vundle</code>安装它, 在.vimrc中添加</p>
<div class="highlight"><pre><span></span><code>Bundle "scrooloose/syntastic"
</code></pre></div>
<p>重新打开Vim, 执行</p>
<div class="highlight"><pre><span></span><code>:BundleInstall
</code></pre></div>
<h3 id="_5">配置</h3>
<p>因为Python已经有<code>pylint</code>来检查, 而且<code>syntastic</code>检查Python会在保存时有很长时间的卡顿, 所以禁用它对Python文件的检查</p>
<div class="highlight"><pre><span></span><code>let g:syntastic_ignore_files=[".*\.py$"]
</code></pre></div>发布一个基于Tornado的高效异步的HTTP客户端库2013-08-01T10:46:00+08:002013-08-01T10:46:00+08:00coldtag:www.linuxzen.com,2013-08-01:/fa-bu-yi-ge-ji-yu-tornadode-gao-xiao-yi-bu-de-httpke-hu-duan-ku.html<p>前面的博文提到过<a href="/shi-yong-tornadojin-xing-wang-luo-yi-bu-bian-cheng.html">使用tornado进行网络异步编程</a>, 也<a href="/jie-yong-tornadoshi-xian-gao-xiao-de-webqqji-qi-ren.html">使用tornado实现了一个高效的WebQQ机器人</a>, 由于tornado内置的<code>AsyncHTTPClient</code>功能过于单一, 所以自己写了一个基于Tornado的HTTP客户端库, 鉴于自己多处使用了这个库, 所以 …</p><p>前面的博文提到过<a href="/shi-yong-tornadojin-xing-wang-luo-yi-bu-bian-cheng.html">使用tornado进行网络异步编程</a>, 也<a href="/jie-yong-tornadoshi-xian-gao-xiao-de-webqqji-qi-ren.html">使用tornado实现了一个高效的WebQQ机器人</a>, 由于tornado内置的<code>AsyncHTTPClient</code>功能过于单一, 所以自己写了一个基于Tornado的HTTP客户端库, 鉴于自己多处使用了这个库, 所以从项目中提取出来, 写成一个单独库 <code>tornadohttpclient</code></p>
<p><s>TornadoHTTPClient 是一个基于Tornado的高效的异步HTTP客户端库, 支持Cookie和代理, 目前仅在<code>Python2.7</code>平台上测试过, 不支持<code>Python3</code></s></p>
<p>听取了仙子君的意见, 直接对<code>tornado.curl_httpclient.CurlAsyncHTTPClient</code>进行封装</p>
<h2 id="_1">安装</h2>
<p>首先从git clone 下代码</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>https://github.com/coldnight/tornadohttpclient.git
</code></pre></div>
<p>然后安装它</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>tornadohttpclient
python<span class="w"> </span>setup.py<span class="w"> </span>install
</code></pre></div>
<h2 id="_2">教程</h2>
<h3 id="get">GET</h3>
<p>TornadoHTTPClient的get方法可以发起一个get请求</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="c1"># 实例化</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="c1"># 发出get请求</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">)</span>
<span class="c1"># 开始主事件循环</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="post">POST</h3>
<p>TornadoHTTPClient的post方法可以发起一个post请求</p>
<h3 id="_3">读取响应</h3>
<p>上面仅仅发出了请求, 但是我们无法读取GET请求回来的数据, 我们可以使用一个回调来读取响应</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<p>通过<code>callback</code>关键字参数我们可以传进一个回调函数, 当请求成功时会调用此函数, 并给此函数传递一个与<code>urllib2.urlopen</code>返回一样的reponse实例</p>
<h3 id="_4">上传文件</h3>
<p><code>upload</code>方法可以上传文件, 其接受一个url和文件的field和文件路径, 还有其他post参数</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"打开图片链接"</span><span class="p">,</span> <span class="n">end</span> <span class="o">=</span> <span class="s2">" "</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">effective_url</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">upload</span><span class="p">(</span><span class="s2">"http://paste.linuxzen.com"</span><span class="p">,</span> <span class="s2">"img"</span><span class="p">,</span> <span class="s2">"img_test.png"</span><span class="p">,</span>
<span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="callback">给callback传递参数</h3>
<p>有时候callback可能需要访问局部变量, 可以通过 <code>args</code>和<code>kwargs</code>关键字参数, 将<code>callback</code>的参数传递给<code>get</code>/<code>post</code>方法, <code>args</code>参数将会在<code>response</code>参数之后被传递,
<code>args</code>参数类型应当是一个元组, <code>kwargs</code>参数类型应当是一个字典</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">,</span> <span class="n">times</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="nb">print</span> <span class="n">times</span>
<span class="k">if</span> <span class="n">times</span> <span class="o">==</span> <span class="mi">9</span><span class="p">:</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="p">))</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="_5">发送延迟请求</h3>
<p>有时我们需要延迟几秒也发送请求或每隔几秒就发送一个请求, <code>get</code>/<code>post</code>方法的<code>delay</code>关键字参数可以解决, <code>delay</code>参数接受一个单位为秒的数字, 并延迟<code>delay</code>秒后发起请求</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">,</span> <span class="n">times</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="k">if</span> <span class="n">times</span> <span class="o"><</span> <span class="mi">9</span><span class="p">:</span>
<span class="c1"># 延迟10秒发送此请求</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="p">(</span><span class="n">times</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="p">),</span> <span class="n">delay</span> <span class="o">=</span> <span class="mi">10</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">))</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="_6">给请求传递参数</h3>
<p>TornadoHTTPClient 的 <code>get</code>/<code>post</code>方法的第二个参数<code>params</code>可以定义请求时传递的参数<code>params</code>的类型为字典或者<code>((key, value), )</code>类型的元组或列表,例如使用百度搜索<code>TornadoHTTPClient</code></p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.baidu.com/s"</span><span class="p">,</span> <span class="p">((</span><span class="s2">"wd"</span><span class="p">,</span> <span class="s2">"tornado"</span><span class="p">),),</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<p>以上也使用与POST方法, 比如登录网站</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="s2">"http://ip.or.domain/login"</span><span class="p">,</span> <span class="p">((</span><span class="s2">"username"</span><span class="p">,</span> <span class="s2">"cold"</span><span class="p">),</span> <span class="p">(</span><span class="s2">"password"</span><span class="p">,</span> <span class="s2">"pwd"</span><span class="p">)),</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="http">指定HTTP头</h3>
<p>TornadoHTTPClient 的<code>get</code>/<code>post</code>方法的 <code>headers</code>关键字参数可以自定额外的HTTP头信息, 参数类型为一个字典</p>
<p>指定User-Agent头</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">headers</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(((</span><span class="s2">"User-Agent"</span><span class="p">,</span>
<span class="s2">"Mozilla/5.0 (X11; Linux x86_64)"</span>\
<span class="s2">" AppleWebKit/537.11 (KHTML, like Gecko)"</span>\
<span class="s2">" Chrome/23.0.1271.97 Safari/537.11"</span><span class="p">),</span> <span class="p">))</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.linuxzen.com"</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
</code></pre></div>
<h3 id="_7">使用代理</h3>
<p>TornadoHTTPClient 的<code>set_proxy</code>方法可以设置代理, 其接受两个参数, 分别是代理的 主机名/ip 代理的端口, <code>unset_proxy</code>可以取消代理</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tornadohttpclient</span> <span class="kn">import</span> <span class="n">TornadoHTTPClient</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">TornadoHTTPClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="n">response</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">response</span><span class="o">.</span><span class="n">body</span>
<span class="n">http</span><span class="o">.</span><span class="n">unset_proxy</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="n">http</span><span class="o">.</span><span class="n">set_proxy</span><span class="p">(</span><span class="s2">"127.0.0.1"</span><span class="p">,</span> <span class="mi">8087</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://shell.appspot.com"</span><span class="p">,</span> <span class="n">callback</span> <span class="o">=</span> <span class="n">callback</span><span class="p">)</span>
<span class="n">http</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<h3 id="cookie">Cookie</h3>
<p>TornadoHTTPClient会自动记录和装载Cookie, 可以通过 TornadoHTTPClient实例属性 cookie 获取Cookie</p>解决git提交敏感信息(回退git版本库到某一个commit)2013-06-07T00:00:00+08:002013-06-07T00:00:00+08:00coldtag:www.linuxzen.com,2013-06-07:/jie-jue-gitti-jiao-min-gan-xin-xi-hui-tui-gitban-ben-ku-dao-mou-yi-ge-commit.html<p>git是一个很好的版本库, 现在很多人用它, 并在github上创建项目, 相信大家都有过将敏感信息提交版本的经历, 如何删除? 好像只有删除版本库来解决, 其实我们 …</p><p>git是一个很好的版本库, 现在很多人用它, 并在github上创建项目, 相信大家都有过将敏感信息提交版本的经历, 如何删除? 好像只有删除版本库来解决, 其实我们可以通过回退版本库删除相应的commit来将提交的敏感信息去掉.</p>
<h2 id="_1">备份本地代码</h2>
<p>首先我们将本地代码的更改备份一下, 以防丢失更改</p>
<h2 id="commit">回退本地代码的commit</h2>
<p>备份完数据, 我们就可以先回退本地的版本库</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>reset<span class="w"> </span>--hard<span class="w"> </span>HEAD~1<span class="w"> </span><span class="c1"># 回退到上一次的提交, 如果是上n次就将1改成对应的数字</span>
</code></pre></div>
<h2 id="_2">回退远端版本库</h2>
<p>接下来如果你直接提交会发现提交不了, 说远端做了更改需要先pull一下, 如果pull咱们就白白做上面的操作, 所以我们可以在别的分支操作</p>
<h3 id="_3">新建一个分支, 并提交</h3>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>checkout<span class="w"> </span>-b<span class="w"> </span>temp
git<span class="w"> </span>push<span class="w"> </span>origin<span class="w"> </span>temp:temp
</code></pre></div>
<h3 id="_4">重建主分支</h3>
<p>下面我们可以删除并重建主分支, 如果是<code>github</code>的话需要将<code>Default Branch</code>切换到别的分支(项目主页->Settings即可看到)</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>push<span class="w"> </span>origin<span class="w"> </span>--delete<span class="w"> </span>master<span class="w"> </span><span class="c1"># 删除远端主分支</span>
git<span class="w"> </span>branch<span class="w"> </span>-d<span class="w"> </span>master<span class="w"> </span><span class="c1"># 删除本地主分支</span>
git<span class="w"> </span>checkout<span class="w"> </span>-b<span class="w"> </span>master<span class="w"> </span><span class="c1"># 新建主分支并切换到主分支</span>
git<span class="w"> </span>push<span class="w"> </span>origin<span class="w"> </span>master<span class="w"> </span><span class="c1"># 提交主分支</span>
</code></pre></div>
<p>这样我们就删除之前提交的敏感信息(如果是<code>github</code>现在就可以把默认分支切换到 <code>master</code>)</p>
<h3 id="_5">删除临时分支</h3>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>branch<span class="w"> </span>-d<span class="w"> </span>temp
git<span class="w"> </span>push<span class="w"> </span>origin<span class="w"> </span>--delete<span class="w"> </span>temp
</code></pre></div>Pual 更新支持SimSimi可以进行互动2013-05-30T09:50:00+08:002013-05-30T09:50:00+08:00coldtag:www.linuxzen.com,2013-05-30:/pual-geng-xin-zhi-chi-simsimike-yi-jin-xing-hu-dong.html<p><a href="http://www.linuxzen.com/jie-yong-tornadoshi-xian-gao-xiao-de-webqqji-qi-ren.html">Pual</a> 跑了许久, 通过一段时间的修改现在Pual主要支持以下功能:</p>
<ul>
<li>英汉互译</li>
<li>为每个用户分配一个session的含有上下文的Python shell</li>
<li>贴代码</li>
</ul>
<hr>
<p>总而言之就是一个被动型的辅助机 …</p><p><a href="http://www.linuxzen.com/jie-yong-tornadoshi-xian-gao-xiao-de-webqqji-qi-ren.html">Pual</a> 跑了许久, 通过一段时间的修改现在Pual主要支持以下功能:</p>
<ul>
<li>英汉互译</li>
<li>为每个用户分配一个session的含有上下文的Python shell</li>
<li>贴代码</li>
</ul>
<hr>
<p>总而言之就是一个被动型的辅助机器人, 群里有同学建议<code>AI</code>功能, 但是我水平不够没办法设计和实现<code>AI</code>部分, 所以想调用<code>SimSimi</code>实现<code>AI</code>, 发现官方Key才免费7天, 我这等穷苦人如何是买不起key的, google发现有一个非官方API可以调用, 但是被封了.但咱不是个容易放弃的人, 经过一番折腾非官方API可以正常调用, 所以Pual也有<code>AI</code>功能了, 只要在有Pual的群里发送<code>Pual</code>打头的消息 就可以和<code>Pual</code>互动</p>
<p><a href="https://github.com/coldnight/pual_bot">项目地址</a></p>
<p>Pual帐号是:1685359365, 大家可以先加好友回答验证问题:cold, 然后将它拉入群内进行调戏</p>说说Python装饰器2013-05-20T14:52:00+08:002013-05-20T14:52:00+08:00coldtag:www.linuxzen.com,2013-05-20:/shuo-shuo-pythonzhuang-shi-qi.html<p>装饰器对与Python新手以至于熟悉Python的人都是一个难理解, 难写的东西. 那么今天就分享一下我对Python 装饰器的理解</p>
<p>所谓装饰器仅仅是一种语法糖, 可作用的对象可以 …</p><p>装饰器对与Python新手以至于熟悉Python的人都是一个难理解, 难写的东西. 那么今天就分享一下我对Python 装饰器的理解</p>
<p>所谓装饰器仅仅是一种语法糖, 可作用的对象可以是函数也可以是类, 装饰器本身是一个函数, 其主要工作方式就是将被装饰的类或者函数当作参数传递给装饰器函数, 比如定义如下装饰器</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">run_time</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">start</span>
<span class="k">return</span> <span class="n">r</span>
<span class="k">return</span> <span class="n">wrapper</span>
</code></pre></div>
<p>我们用这个装饰器装饰一个<code>test</code>函数</p>
<div class="highlight"><pre><span></span><code><span class="nd">@run_time</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="nb">print</span> <span class="s2">"just a test"</span>
</code></pre></div>
<p>前面说过其实装饰器就是一个语法糖, 就是将被装饰的函数作为参数传递给装饰器函数, 所以上面可以展开为</p>
<div class="highlight"><pre><span></span><code><span class="n">test</span> <span class="o">=</span> <span class="n">run_time</span><span class="p">(</span><span class="n">test</span><span class="p">)</span>
</code></pre></div>
<p>装饰器将在解释器运行一开始就被加载, 从而将被装饰的函数将被展开成如上方式, 因为 <code>run_time</code>装饰器返回<code>wrapper</code>函数, 所以当调用<code>test</code>函数时其实就是对<code>wrapper</code>的调用</p>
<p>如果你在Python shell下执行以上语句就会发现定义完<code>test</code>函数然后查看<code>test</code>时, shell所展示的是wrapper函数:
<a href="http://www.linuxzen.com"><img alt="说说Python装饰器" src="/static/upload/pyshell.png"></a></p>
<hr>
<p>接下来说说如何编写带参数的装饰器, 大家如果细心的话就可以发现其实带参数的装饰器是经过调用"装饰器"函数返回的一个装饰器, 之所以装饰器上打引号是说明其实这个所谓的"装饰器"只不过是一个普通的函数, 但这个普通的函数返回一个装饰器, 可以参看下面例子:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">route</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="n">func</span><span class="o">.</span><span class="n">__url__</span> <span class="o">=</span> <span class="n">url</span>
<span class="k">return</span> <span class="n">func</span>
<span class="k">return</span> <span class="n">decorator</span>
<span class="nd">@route</span><span class="p">(</span><span class="sa">r</span><span class="s2">"/"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">index</span><span class="p">():</span>
<span class="k">return</span> <span class="s2">"Hi"</span>
</code></pre></div>
<p>大家可以发现在使用<code>route</code>装饰器时我们其实是调用了<code>route</code>函数, <code>route</code>函数返回一个<code>decorator</code>装饰器, 因为我们不需要在装饰器内运行函数, 所以不需要一个<code>wrapper</code>函数来收集参数.</p>
<hr>
<p>以上就是全部内容, 希望对装饰器一知半解的人有些许帮助</p>clubot更新: 使用SQLAlchemy重写数据库部分和改用Tornado MainLoop2013-04-26T15:40:00+08:002013-04-26T15:40:00+08:00coldtag:www.linuxzen.com,2013-04-26:/clubotgeng-xin-shi-yong-sqlalchemyzhong-xie-shu-ju-ku-bu-fen-he-gai-yong-tornado-mainloop.html<p><a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>在我的vps上跑了有一段时间了, 最近接触了<code>SQLAlchemy</code> 然后反观<code>clubot</code>的数据库代码部分, 感觉代码又遭有乱实在看不过眼, 所以就使用<code>SQLAlchemy</code>重写了数据库 …</p><p><a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>在我的vps上跑了有一段时间了, 最近接触了<code>SQLAlchemy</code> 然后反观<code>clubot</code>的数据库代码部分, 感觉代码又遭有乱实在看不过眼, 所以就使用<code>SQLAlchemy</code>重写了数据库模块, 并将<code>epoll</code>的MainLoop改成<a href="http://lilydjwg.is-programmer.com/">仙子君</a>所写的<a href="https://github.com/lilydjwg/pyxmpp2">TornadoMainLoop</a></p>
<h2 id="_1">更新内容</h2>
<ol>
<li>数据库使用<code>SQLAlchemy</code>重写</li>
<li><code>MainLoop</code>改用<code>TornadoMainLoop</code></li>
<li>改变代码结构, 清理部分代码</li>
<li>将<code>history</code>命令改为<code>old</code>, 并支持时间查询</li>
<li>废弃一些不常用的命令</li>
<li>改变数据库表结构</li>
<li>废弃<code>channel</code>功能, <code>cd</code>命令仅支持切换聊天和安静模式</li>
<li>删除一些不用的配置</li>
</ol>
<h2 id="_2">如何升级</h2>
<p>数据库表结构做了更改, 所以为了兼容之前的数据库本次表名前加上<code>clubot_</code>前缀, 并配以<code>update.py</code>脚本用以支持将旧的数据导入.</p>
<h2 id="_3">新的依赖</h2>
<p>本次更新添加了依赖, 现在依赖包括:</p>
<ul>
<li>pyxmpp2</li>
<li>dnspython</li>
<li>tornado</li>
<li>sqlalchemy</li>
<li>MySQL-python</li>
</ul>
<h2 id="_4">乱码</h2>
<p>如果数据导入后乱码, 可以参考<a href="/sqlalchemy-mysqlshu-ju-ku-luan-ma-jie-jue.html">这篇文章</a></p>
<h2 id="_5">项目地址</h2>
<p>最后放上<a href="https://github.com/coldnight/clubot">项目地址</a></p>SQLAlchemy MySQL数据库乱码解决2013-04-26T00:00:00+08:002013-04-26T00:00:00+08:00coldtag:www.linuxzen.com,2013-04-26:/sqlalchemy-mysqlshu-ju-ku-luan-ma-jie-jue.html<p>今天对<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>进行了<a href="/clubotgeng-xin-shi-yong-sqlalchemyzhong-xie-shu-ju-ku-bu-fen-he-gai-yong-tornado-mainloop.html">升级</a>, 但是导入数据后中文乱码, 一开是找资料说是在创建引擎的时候添加编码信息:</p>
<div class="highlight"><pre><span></span><code><span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"mysql://root:@localhost:3306/clubot?charset …</span></code></pre></div><p>今天对<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>进行了<a href="/clubotgeng-xin-shi-yong-sqlalchemyzhong-xie-shu-ju-ku-bu-fen-he-gai-yong-tornado-mainloop.html">升级</a>, 但是导入数据后中文乱码, 一开是找资料说是在创建引擎的时候添加编码信息:</p>
<div class="highlight"><pre><span></span><code><span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"mysql://root:@localhost:3306/clubot?charset=utf8"</span><span class="p">)</span>
</code></pre></div>
<p>但是这并不行, 然后查看表信息:</p>
<div class="highlight"><pre><span></span><code><span class="o">></span><span class="w"> </span><span class="k">show</span><span class="w"> </span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">clubot_members</span><span class="p">;</span>
<span class="n">clubot_members</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n n-Quoted">`clubot_members`</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n n-Quoted">`id`</span><span class="w"> </span><span class="kt">int</span><span class="p">(</span><span class="mi">11</span><span class="p">)</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="no">NULL</span><span class="w"> </span><span class="k">AUTO_INCREMENT</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`email`</span><span class="w"> </span><span class="kt">varchar</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`nick`</span><span class="w"> </span><span class="kt">varchar</span><span class="p">(</span><span class="mi">50</span><span class="p">)</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`last_say`</span><span class="w"> </span><span class="kt">timestamp</span><span class="w"> </span><span class="no">NULL</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`last_change`</span><span class="w"> </span><span class="kt">timestamp</span><span class="w"> </span><span class="no">NULL</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`isonline`</span><span class="w"> </span><span class="kt">int</span><span class="p">(</span><span class="mi">11</span><span class="p">)</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="n n-Quoted">`join_date`</span><span class="w"> </span><span class="kt">timestamp</span><span class="w"> </span><span class="no">NULL</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="no">NULL</span><span class="p">,</span>
<span class="w"> </span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="p">(</span><span class="n n-Quoted">`id`</span><span class="p">),</span>
<span class="w"> </span><span class="k">UNIQUE</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="n n-Quoted">`email`</span><span class="w"> </span><span class="p">(</span><span class="n n-Quoted">`email`</span><span class="p">),</span>
<span class="w"> </span><span class="k">UNIQUE</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="n n-Quoted">`nick`</span><span class="w"> </span><span class="p">(</span><span class="n n-Quoted">`nick`</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span><span class="k">ENGINE</span><span class="o">=</span><span class="n">InnoDB</span><span class="w"> </span><span class="k">AUTO_INCREMENT</span><span class="o">=</span><span class="mi">20</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="k">CHARSET</span><span class="o">=</span><span class="n">latin1</span><span class="p">;</span>
</code></pre></div>
<p>发现原来创建表的时候用的<code>latin1</code>编码, 而老的表是用<code>utf-8</code>编码创建的, <code>SQLAlchemy</code>中并没有发现有创建表时指定指定编码的方法. 所以只能在<code>MySQL</code>本身来找:</p>
<div class="highlight"><pre><span></span><code><span class="o">></span><span class="w"> </span><span class="k">show</span><span class="w"> </span><span class="k">VARIABLES</span><span class="w"> </span><span class="k">like</span><span class="w"> </span><span class="s2">"character%%"</span><span class="p">;</span>
<span class="o">+--------------------------+-----------------------------+</span>
<span class="o">|</span><span class="w"> </span><span class="n">Variable_name</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Value</span><span class="w"> </span><span class="o">|</span>
<span class="o">+--------------------------+-----------------------------+</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_client</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">utf8</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_connection</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">utf8</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_database</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">latin1</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_filesystem</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="kt">binary</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_results</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">utf8</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_server</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">latin1</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_set_system</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">utf8</span><span class="w"> </span><span class="o">|</span>
<span class="o">|</span><span class="w"> </span><span class="n">character_sets_dir</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">/</span><span class="k">data</span><span class="o">/</span><span class="k">share</span><span class="o">/</span><span class="n">mysql</span><span class="o">/</span><span class="n">charsets</span><span class="o">/</span><span class="w"> </span><span class="o">|</span>
<span class="o">+--------------------------+-----------------------------+</span>
<span class="mi">8</span><span class="w"> </span><span class="k">rows</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="kt">set</span><span class="w"> </span><span class="p">(</span><span class="mf">0.00</span><span class="w"> </span><span class="n">sec</span><span class="p">)</span>
<span class="o">></span><span class="w"> </span><span class="k">show</span><span class="w"> </span><span class="k">create</span><span class="w"> </span><span class="k">database</span><span class="w"> </span><span class="n">clubot</span><span class="p">;</span>
<span class="o">+----------+-------------------------------------------------------------------+</span>
<span class="o">|</span><span class="w"> </span><span class="k">Database</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Create</span><span class="w"> </span><span class="k">Database</span><span class="w"> </span><span class="o">|</span>
<span class="o">+----------+-------------------------------------------------------------------+</span>
<span class="o">|</span><span class="w"> </span><span class="n">clubot</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">CREATE</span><span class="w"> </span><span class="k">DATABASE</span><span class="w"> </span><span class="n n-Quoted">`clubot`</span><span class="w"> </span><span class="cm">/*!40100 DEFAULT CHARACTER SET latin1 */</span><span class="w"> </span><span class="o">|</span>
<span class="o">+----------+-------------------------------------------------------------------+</span>
<span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="kt">set</span><span class="w"> </span><span class="p">(</span><span class="mf">0.00</span><span class="w"> </span><span class="n">sec</span><span class="p">)</span>
</code></pre></div>
<p>发现 <code>MySQL</code>默认的和数据库都是<code>latin1</code>的编码, 所以更改数据库配置</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/mysql/my.cnf<span class="w"> </span><span class="c1"># MySQL配置文件在Ubuntu上的位置, 其他系统可能有差异</span>
</code></pre></div>
<p>分别在<code>[client]</code> <code>[mysqld]</code>下添加</p>
<div class="highlight"><pre><span></span><code>default-character-set = utf8
</code></pre></div>
<p>这时重启MySQL居然起不来, 说<code>default-character-set</code>是无效的变量, 查看<code>MySQL</code>版本发现是5.5, 找资料说5.5的服务端编码设置变量是<code>character-set-server</code>, 所以将<code>[mysqld]</code>上的<code>default-character-set = utf8</code>改为 <code>character-set-server = utf8</code>, 并重启<code>MySQL</code></p>
<p>然后更改数据库编码:</p>
<div class="highlight"><pre><span></span><code><span class="k">alter</span><span class="w"> </span><span class="k">database</span><span class="w"> </span><span class="n">clubot</span><span class="w"> </span><span class="k">character</span><span class="w"> </span><span class="k">set</span><span class="w"> </span><span class="n">utf8</span><span class="p">;</span>
</code></pre></div>
<p>删除新建的表, 并重新导入数据中文就正常了</p>
<div class="highlight"><pre><span></span><code><span class="o">></span><span class="w"> </span><span class="k">use</span><span class="w"> </span><span class="n">clubot</span><span class="p">;</span>
<span class="o">></span><span class="w"> </span><span class="k">drop</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">clubot_status</span><span class="p">;</span>
<span class="o">></span><span class="w"> </span><span class="k">drop</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">clubot_infos</span><span class="p">;</span>
<span class="o">></span><span class="w"> </span><span class="k">drop</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">clubot_history</span><span class="p">;</span>
<span class="o">></span><span class="w"> </span><span class="k">drop</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">clubot_members</span><span class="p">;</span>
</code></pre></div>grep挽救了我一个下午: 恢复rm -f删除的代码2013-04-25T14:36:00+08:002013-04-25T14:36:00+08:00coldtag:www.linuxzen.com,2013-04-25:/grepwan-jiu-liao-wo-yi-ge-xia-wu-hui-fu-rm-fshan-chu-de-dai-ma.html<p>今天对代码进行重构, 新添加了一个<code>logics.py</code>模块, 但并没有加入到git库中, 然后对代码进行清理, 有一个<code>logs</code>文件夹是不需要的所以 …</p><p>今天对代码进行重构, 新添加了一个<code>logics.py</code>模块, 但并没有加入到git库中, 然后对代码进行清理, 有一个<code>logs</code>文件夹是不需要的所以我打算清掉它, 但是<code>zsh</code>将补全定位到<code>logics.py</code>, 手上的动作快过了脑子,直接按了回车, 做完心想完了, 一上午白费了, 还要花一下午的时间来重新写出这个代码, 虽然明知没有希望,但是我还是不想放弃, 于是就google一下有无解决办法,意外的是找到一篇文章:<a href="http://www.vpsee.com/2010/08/using-grep-to-recover-text-files/">用grep恢复误删的文本</a></p>
<p>好吧虽然不知道能不能用, 但是我要试试, 因为我不想再花一个下午来重写这个模块, 我在模块里定义了一个<code>Logics</code>类并且上面差不多30行, 往下差不多300行的样子, 我删除<code>logics.py</code>所在的分区是<code>/dev/sda7</code>, 所以我运行了下面命令:</p>
<div class="highlight"><pre><span></span><code>grep<span class="w"> </span>-a<span class="w"> </span>-B<span class="w"> </span><span class="m">50</span><span class="w"> </span>-A<span class="w"> </span><span class="m">400</span><span class="w"> </span><span class="s2">"class Logics(object):"</span><span class="w"> </span>/dev/sda7<span class="w"> </span>><span class="w"> </span>tmp
</code></pre></div>
<p>经过漫长的等待, 这个命令还没结束, 我忍不住看了下 tmp的内容, 我发现我找到了这段代码</p>
<p>当然损失并不是没有, 损失就是中文注释全部乱码, 但我满足了, 我只要重写注释就可以了.</p>借用Tornado实现高效的WebQQ机器人2013-04-23T13:36:00+08:002013-04-23T13:36:00+08:00coldtag:www.linuxzen.com,2013-04-23:/jie-yong-tornadoshi-xian-gao-xiao-de-webqqji-qi-ren.html<p>之前有写过一篇文章介绍使用<code>Pyxmpp2</code>桥接QQ和xmpp的文章(<a href="/shi-yong-webqqxie-yi-qiao-jie-xmpphe-qqqun.html">这里</a>).后来我打算将WebQQ单独出来运行, 一开始直接拷贝了<code>pyxmpp2</code>的mainloop, 但是跑起来问题多多, 所以我又 …</p><p>之前有写过一篇文章介绍使用<code>Pyxmpp2</code>桥接QQ和xmpp的文章(<a href="/shi-yong-webqqxie-yi-qiao-jie-xmpphe-qqqun.html">这里</a>).后来我打算将WebQQ单独出来运行, 一开始直接拷贝了<code>pyxmpp2</code>的mainloop, 但是跑起来问题多多, 所以我又研究了利用<code>Tornado</code>进行网络编程(<a href="/shi-yong-tornadojin-xing-wang-luo-yi-bu-bian-cheng.html">这里</a>), 所以我放弃了<code>Pyxmpp2</code>的mainloop,使用<code>Tornado</code>进行重写</p>
<p>首先放出<a href="https://github.com/coldnight/pual_bot">项目代码</a></p>
<h2 id="_1">引子</h2>
<p>WebQQ协议是一套基于<code>HTTP</code>的QQ协议, 而用<code>Python</code>的<code>urllib2</code>库进行请求太慢, 因为HTTP本身就使用socket请求, 所以改用多路复用I/O模型, 而<code>Tornado</code>简单高效, 看过代码后可以轻松上手.平台兼容性很好, 所以选择<code>Tornado</code>作为网络框架.</p>
<h2 id="_2">原理</h2>
<p>首先实现了一个 <code>HTTPStream</code>类, 其主要接口是<code>add_request</code>方法, 它接受一个必选参数:<code>request</code> 是一个 <code>urllib2.Request</code>的实例, 和一个可选参数:<code>readback</code>是一个接受一个<code>urllib2.urlopen(request)</code>返回的<code>Response</code>参数的读取函数, 代码如下:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">HTTPStream</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="c1"># 省略若干代码</span>
<span class="k">def</span> <span class="nf">add_request</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">readback</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">):</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">,</span> <span class="s2">"Not a invaid requset"</span>
<span class="c1"># 此处易触发timeout异常, 省略处理异常代码</span>
<span class="n">sock</span><span class="p">,</span> <span class="n">data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http_sock</span><span class="o">.</span><span class="n">make_http_sock_data</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>
<span class="n">fd</span> <span class="o">=</span> <span class="n">sock</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">fd_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span> <span class="o">=</span> <span class="n">sock</span>
<span class="bp">self</span><span class="o">.</span><span class="n">fd_request_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span> <span class="o">=</span> <span class="n">request</span>
<span class="n">callback</span> <span class="o">=</span> <span class="n">partial</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_handle_events</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">readback</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">add_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">callback</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">WRITE</span><span class="p">)</span>
</code></pre></div>
<p><code>HTTPStream.add_request</code>将<code>urllib2.Request</code>的实例解析出一个<code>socket</code>和一个用于<code>socket</code>发送的数据.前面文章介绍过了, <code>tornado.ioloop.IOLoop.add_handler</code>用于将注册socket, 其需要三个参数: socket的文件描述符, 接受文件描述符和事件参数的回调, 和注册的事件.</p>
<p>我们用到的回调是<code>HTTPStream._handle_events</code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">HTTPStream</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="c1"># 省略若干代码</span>
<span class="k">def</span> <span class="nf">_handle_events</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">readback</span><span class="p">,</span> <span class="n">fd</span><span class="p">,</span> <span class="n">event</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 用于处理Tornado事件</span>
<span class="sd"> Arguments:</span>
<span class="sd"> `request` - urllib.Request</span>
<span class="sd"> `data` - socket要写入的数据</span>
<span class="sd"> `readback` - 读取函数</span>
<span class="sd"> 以上参数应当使用partial封装然后将此方法作为IOLoop.add_handler的callback</span>
<span class="sd"> `fd` - IOLoop传递 文件描述符</span>
<span class="sd"> `event` - IOLoop传递 tornado</span>
<span class="sd"> """</span>
<span class="n">s</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fd_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">:</span>
<span class="c1"># 省略错误处理</span>
<span class="n">resp</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">http_sock</span><span class="o">.</span><span class="n">make_response</span><span class="p">(</span><span class="n">s</span><span class="p">,</span> <span class="n">request</span><span class="p">)</span>
<span class="n">args</span> <span class="o">=</span> <span class="n">readback</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span>
<span class="k">if</span> <span class="n">args</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">add_delay_request</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="n">args</span><span class="p">)</span>
<span class="n">t</span><span class="o">.</span><span class="n">setDaemon</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
<span class="n">t</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="k">if</span> <span class="n">args</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_request</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">remove_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">WRITE</span><span class="p">:</span>
<span class="n">s</span><span class="o">.</span><span class="n">sendall</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">if</span> <span class="n">readback</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">update_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ioloop</span><span class="o">.</span><span class="n">remove_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">ERROR</span><span class="p">:</span>
<span class="k">pass</span>
</code></pre></div>
<p>它接受的参数上面注释写的很清楚, 不做解释, 所以将此方法通过<code>functools.partial</code>封装做为<code>callback</code>传递给<code>tornado.ioloop.IOLoop.add_handler</code>, 并注册为<code>写</code>事件, 以便发送<code>HTTP</code>请求.</p>
<p><code>HTTPStream._handle_events</code>用于处理事件, 当事件为写时就发送<code>HTTP</code>请求(根据<code>urllib2.Request</code>生成的用于发送的数据), 并判断是否有读取函数, 有则注册<code>读</code>事件, 当事件为读时就从socket中构建一个<code>Response</code>并传递给读取函数, 读取函数会返回3个值, 分别为: 下一个请求, 请求的读取函数(可为None, 为None则只请求不读取), 下一个请求的延迟(多长事件后添加此请求, 可选, 单位为秒)</p>
<p>依据读取函数返回的三个值来确定下一个请求, 并完成一系列的请求. 更加完整的代码请参见文章开头给出的项目代码</p>
<p><code>HTTPStream.http_sock.make_response</code>执行时会将<code>socket</code>设为阻塞, 因为不设置阻塞会出现<code>httplib.BadStatusLine</code>异常.读取函数执行完毕,重新将<code>socket</code>设置为非阻塞, 并移除此<code>socket</code>(虽然做了这样的处理但是QQ连接时间稍长还是会触发<code>httplib.BadStatusLine</code>异常)</p>
<h2 id="2013-04-26">2013-04-26 更新</h2>
<ul>
<li>解决 在线时间稍长, 当经过多次请求后会触发<code>socket.gaierror(-2, 'Name or service not known')</code> 异常</li>
</ul>
<h2 id="_3">存在问题</h2>
<ol>
<li>没有重试机制</li>
</ol>使用Pelican打造静态博客2013-04-18T09:40:00+08:002013-04-18T09:40:00+08:00coldtag:www.linuxzen.com,2013-04-18:/shi-yong-pelicanda-zao-jing-tai-bo-ke.html<p>前面有文章介绍本站采用了<code>Python</code>编写的<code>Pelican</code>静态生成博客系统, 之所以没有使用当前很火的<code>Jekyll</code>, 是因为它是<code>Ruby</code>编写, 而我又对 …</p><p>前面有文章介绍本站采用了<code>Python</code>编写的<code>Pelican</code>静态生成博客系统, 之所以没有使用当前很火的<code>Jekyll</code>, 是因为它是<code>Ruby</code>编写, 而我又对<code>Ruby</code>没有啥兴趣, 所以还是选择了使用了我熟悉的Python编写的这套系统, 我用了一段时间,打算将使用经验分享出来</p>
<h2 id="_1">介绍</h2>
<p><code>Pelican</code>是一套开源的使用Python编写的博客静态生成, 可以添加文章和和创建页面, 可以使用<code>MarkDown</code> <code>reStructuredText</code> 和 <code>AsiiDoc</code> 的格式来抒写, 同时使用 <code>Disqus</code>评论系统, 支持 <code>RSS</code>和<code>Atom</code>输出, 插件, 主题, 代码高亮等功能, 采用<code>Jajin2</code>模板引擎, 可以很容易的更改模板</p>
<h2 id="_2">安装</h2>
<p>可以从<code>github</code>克隆最新的代码安装, 并且建议在<code>virtualenv</code>下使用:</p>
<h3 id="virtualenv">建立 virtualenv</h3>
<div class="highlight"><pre><span></span><code>virtualenv<span class="w"> </span>pelican<span class="w"> </span><span class="c1"># 创建</span>
<span class="nb">cd</span><span class="w"> </span>pelican
sh<span class="w"> </span>bin/activate<span class="w"> </span><span class="c1"># 激活虚拟环境</span>
</code></pre></div>
<p>上面建立了一个Python的虚拟环境(这个命令不是内置可以使用 <code>easy_install virtualenv</code> 安装)</p>
<h3 id="githubpelican">从github克隆最新代码安装Pelican</h3>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git://github.com/getpelican/pelican.git<span class="w"> </span><span class="c1"># 代码</span>
<span class="nb">cd</span><span class="w"> </span>pelican
python<span class="w"> </span>setup.py<span class="w"> </span>install
</code></pre></div>
<p>上面步骤完成后就安装了Pelican</p>
<h2 id="_3">开始一个博客</h2>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>/path/to/your/blog
<span class="nb">cd</span><span class="w"> </span>/path/to/your/blog
pelican-quickstart
</code></pre></div>
<p>在回答一系列问题过后你的博客就建成的, 主要生成下列文件:</p>
<div class="highlight"><pre><span></span><code>.
<span class="p">|</span>--<span class="w"> </span>content<span class="w"> </span><span class="c1"># 所有文章放于此目录</span>
<span class="p">|</span>--<span class="w"> </span>develop_server.sh<span class="w"> </span><span class="c1"># 用于开启测试服务器</span>
<span class="p">|</span>--<span class="w"> </span>Makefile<span class="w"> </span><span class="c1"># 方便管理博客的Makefile</span>
<span class="p">|</span>--<span class="w"> </span>output<span class="w"> </span><span class="c1"># 静态生成文件</span>
<span class="p">|</span>--<span class="w"> </span>pelicanconf.py<span class="w"> </span><span class="c1"># 配置文件</span>
<span class="p">|</span>--<span class="w"> </span>publishconf.py<span class="w"> </span><span class="c1"># 配置文件</span>
</code></pre></div>
<h3 id="_4">写一篇文章</h3>
<p>在 <code>content</code> 目录新建一个 <code>test.md</code>文件, 填入一下内容:</p>
<div class="highlight"><pre><span></span><code>Title: 文章标题
Date: 2013-04-18
Category: 文章类别
Tag: 标签1, 标签2
这里是内容
</code></pre></div>
<p>然后执行:</p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html
</code></pre></div>
<p>用以生成html</p>
<p>然后执行</p>
<div class="highlight"><pre><span></span><code>./develop_server.sh<span class="w"> </span>start
</code></pre></div>
<p>开启一个测试服务器, 这会在本地 8000 端口建立一个测试web服务器, 可以使用浏览器打开:<code>http://localhost:8000</code>来访问这个测试服务器, 然后就可以欣赏到你的博客了</p>
<h3 id="_5">创建一个页面</h3>
<p>这里以创建 <code>About</code>页面为例</p>
<p>在<code>content</code>目录创建<code>pages</code>目录</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>content/pages
</code></pre></div>
<p>然后创建<code>About.md</code>并填入下面内容</p>
<div class="highlight"><pre><span></span><code>Title:<span class="w"> </span>About<span class="w"> </span>Me
Date:<span class="w"> </span><span class="m">2013</span>-04-18
About<span class="w"> </span>me<span class="w"> </span>content
</code></pre></div>
<p>执行 <code>make html</code> 生成html, 然后打开 <code>http://localhost:8000</code>查看效果</p>
<h3 id="pelican">让Pelican支持评论</h3>
<p>Pelican 使用<code>Disqus</code>评论, 可以申请在<a href="https://disqus.com/">Disqus</a>上申请一个站点, 然后在<code>pelicanconf.py</code>里添加或修改<code>DISQUS_SITENAME</code>项:</p>
<div class="highlight"><pre><span></span><code><span class="n">DISQUS_SITENAME</span> <span class="o">=</span> <span class="sa">u</span><span class="s2">"linuxzen"</span>
</code></pre></div>
<p>执行</p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html
</code></pre></div>
<p>浏览器打开 <code>http://localhost:8000</code>查看效果</p>
<h3 id="_6">更换主题</h3>
<p>Pelican本身也提供了一些主题可供选择, 可以从github克隆下来</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git://github.com/getpelican/pelican-themes.git<span class="w"> </span><span class="c1"># 主题</span>
</code></pre></div>
<p>然后在里面找到想要的主题, 然后拷到博客项目当前目录, 这里已<code>neat</code>为例</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>-r<span class="w"> </span>/path/to/themes/from/github/neat<span class="w"> </span>.
</code></pre></div>
<p>然后在 <code>pelicanconf.py</code> 配置文件里添加或修改 <code>THEME</code>项为 <code>neat</code></p>
<div class="highlight"><pre><span></span><code><span class="n">THEME</span> <span class="o">=</span> <span class="s2">"neat"</span>
</code></pre></div>
<p>重新执行 </p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html
</code></pre></div>
<p>然后打开 <code>http://localhost:8000</code> 查看效果</p>
<h3 id="_7">使用插件</h3>
<p>Pelican 一开始是将插件内置的, 但是新版本 Pelican将插件隔离了出来, 所以我们要到github上 克隆一份新的插件, 在博客目录执行</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git://github.com/getpelican/pelican-plugins.git<span class="w"> </span><span class="c1"># 插件</span>
</code></pre></div>
<p>现在我们博客目录就新添了一个 <code>pelican-plugins</code>目录, 我们已配置<code>sitemap</code>插件为例,
<code>sitemap</code>插件可以生成 <code>sitemap.xml</code> 供搜索引擎使用</p>
<p>在<code>pelicanconf.py</code>配置文件里加上如下项:</p>
<div class="highlight"><pre><span></span><code><span class="n">PLUGIN_PATH</span> <span class="o">=</span> <span class="sa">u</span><span class="s2">"pelican-plugins"</span>
<span class="n">PLUGINS</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"sitemap"</span><span class="p">]</span>
<span class="c1">## 配置sitemap 插件</span>
<span class="n">SITEMAP</span> <span class="o">=</span> <span class="p">{</span>
<span class="s2">"format"</span><span class="p">:</span> <span class="s2">"xml"</span><span class="p">,</span>
<span class="s2">"priorities"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"articles"</span><span class="p">:</span> <span class="mf">0.7</span><span class="p">,</span>
<span class="s2">"indexes"</span><span class="p">:</span> <span class="mf">0.5</span><span class="p">,</span>
<span class="s2">"pages"</span><span class="p">:</span> <span class="mf">0.3</span><span class="p">,</span>
<span class="p">},</span>
<span class="s2">"changefreqs"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"articles"</span><span class="p">:</span> <span class="s2">"monthly"</span><span class="p">,</span>
<span class="s2">"indexes"</span><span class="p">:</span> <span class="s2">"daily"</span><span class="p">,</span>
<span class="s2">"pages"</span><span class="p">:</span> <span class="s2">"monthly"</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>然后再执行</p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html
</code></pre></div>
<p>打开浏览器请求 <code>http://localhost:8000/sitemap.xml</code>即可看到生成的 Sitemap 了</p>
<h3 id="_8">拷贝静态文件</h3>
<p>如果我们定义静态的文件, 该如何将它在每次生成的时候拷贝到 output 目录呢, 我们以<code>robots.txt</code> 为例, 在我们的 content/extra 下面我们放了一个定义好的 <code>robots.txt</code>文件, 在<code>pelicanconf.py</code>更改或添加 <code>FILES_TO_COPY</code>项:</p>
<div class="highlight"><pre><span></span><code><span class="n">FILES_TO_COPY</span> <span class="o">=</span> <span class="p">(</span>
<span class="p">(</span><span class="s2">"extra/robots.txt"</span><span class="p">,</span> <span class="s2">"robots.txt"</span><span class="p">),</span>
<span class="p">)</span>
</code></pre></div>
<p>这样在每次生成html的时候都会把 <code>content/extra</code>下的 <code>robots.txt</code> 拷贝到 <code>output</code>目录下</p>
<h3 id="_9">拷贝静态目录</h3>
<p>如果是一个静目录怎么办? 我们可以在<code>pelicanconf.py</code>里添加或修改 <code>STATIC_PATHS</code>项, 比如我们有个<code>img</code>目录用来放文章所使用的图片, 我们可以在<code>pelicanconf.py</code>添加</p>
<div class="highlight"><pre><span></span><code><span class="n">STATIC_PATHS</span> <span class="o">=</span> <span class="p">[</span><span class="sa">u</span><span class="s2">"img"</span><span class="p">]</span>
</code></pre></div>
<p>然后执行 </p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html
</code></pre></div>
<p>然后 Pelican 就会将 <code>img</code>目录拷贝到 <code>output/static/</code> 下</p>
<h2 id="_10">部署</h2>
<p>上面都弄完之后你就可以得到一个功能健全的博客系统, 接下来就是部署到服务器, 上传到服务器并结合nginx或者apache等web服务器部署这里就不在详述</p>
<h2 id="_11">参考</h2>
<p>如果还有其他问题请参考<a href="http://docs.getpelican.com/">官方手册</a></p>使用Tornado进行网络异步编程2013-04-15T16:09:00+08:002013-04-15T16:09:00+08:00coldtag:www.linuxzen.com,2013-04-15:/shi-yong-tornadojin-xing-wang-luo-yi-bu-bian-cheng.html<h2 id="tornado">Tornado</h2>
<p><code>Tornado</code> 是一款非阻塞可扩展的使用Python编写的web服务器和Python Web框架, 可以使用<code>Tornado</code>编写Web程序并不依赖任何web服务器直接提供高效的web服务.所以<code>Tornado</code>不仅仅是一个web框架而且还是一款可以用于生产环境的高效的web服务器</p>
<p>Torando 在Linux和FreeBSD上使用高效的异步I/O模型 <code>epoll</code> 和<code>kqueue</code>来实现高效的web服务器, 所以 …</p><h2 id="tornado">Tornado</h2>
<p><code>Tornado</code> 是一款非阻塞可扩展的使用Python编写的web服务器和Python Web框架, 可以使用<code>Tornado</code>编写Web程序并不依赖任何web服务器直接提供高效的web服务.所以<code>Tornado</code>不仅仅是一个web框架而且还是一款可以用于生产环境的高效的web服务器</p>
<p>Torando 在Linux和FreeBSD上使用高效的异步I/O模型 <code>epoll</code> 和<code>kqueue</code>来实现高效的web服务器, 所以 tornado在Linux上和FreeBSD系列性能可以达到最高</p>
<h2 id="_1">接口</h2>
<p>当然我们可以不仅仅把<code>Tornado</code>看作是一个web框架和web服务器, 我们可以利用<code>Tornado</code>提供的接口进行高效的网络异步编程,</p>
<p><code>tornado.ioloop.IOLoop</code> 提供了三个接口可以用于网络编程:</p>
<h3 id="add_handler">add_handler</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">add_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fd</span><span class="p">,</span> <span class="n">handler</span><span class="p">,</span> <span class="n">events</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span> <span class="o">=</span> <span class="n">stack_context</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_impl</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">events</span> <span class="o">|</span> <span class="bp">self</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span>
</code></pre></div>
<p><code>add_handler</code>用于添加socket到主循环中, 接受三个参数:
* fd 是socket的文件描述符
* handler 是处理此socket的 callback函数
* events 是此socket注册的事件</p>
<h3 id="update_handler">update_handler</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">update_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fd</span><span class="p">,</span> <span class="n">events</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_impl</span><span class="o">.</span><span class="n">modify</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">events</span> <span class="o">|</span> <span class="bp">self</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span>
</code></pre></div>
<p><code>update_handler</code>用于更新住循环中已存在的socket响应事件, 接受两个参数:
* fd 是socket对应的文件描述符
* events 是注册的新事件</p>
<h3 id="remove_handler">remove_handler</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">remove_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fd</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_events</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_impl</span><span class="o">.</span><span class="n">unregister</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
<span class="n">gen_log</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Error deleting fd from IOLoop"</span><span class="p">,</span> <span class="n">exc_info</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>
<p><code>remove_handler</code>用于移除主循环中已存在的socket</p>
<h2 id="_2">事件</h2>
<p><code>tornado.ioloop.IOLoop</code>同时提供了4种响应事件:</p>
<table>
<thead>
<tr>
<th>事件</th>
<th>描述</th>
</tr>
</thead>
<tbody>
<tr>
<td>tornado.ioloop.IOLoop.NONE</td>
<td>无事件</td>
</tr>
<tr>
<td>tornado.ioloop.IOLoop.READ</td>
<td>读事件</td>
</tr>
<tr>
<td>tornado.ioloop.IOLoop.WRITE</td>
<td>写事件</td>
</tr>
<tr>
<td>tornado.ioloop.IOLoop.ERROR</td>
<td>发生错误的事件</td>
</tr>
</tbody>
</table>
<h2 id="_3">实例</h2>
<p>根据上面的接口和事件我们就可以写出一个简单的 echo server</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 13/04/15 15:08:51</span>
<span class="c1"># Desc : Tornado Echo Server</span>
<span class="c1"># HOME : http://www.linuxzen.com</span>
<span class="c1">#</span>
<span class="kn">import</span> <span class="nn">Queue</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">partial</span>
<span class="kn">from</span> <span class="nn">tornado.ioloop</span> <span class="kn">import</span> <span class="n">IOLoop</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="o">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="o">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="c1"># 将socket设置为非阻塞</span>
<span class="n">server_address</span> <span class="o">=</span> <span class="p">(</span><span class="s2">"localhost"</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="n">server_address</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">listen</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="n">fd_map</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># 文件描述符到socket的映射</span>
<span class="n">message_queue_map</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># socket到消息队列的映射</span>
<span class="n">fd</span> <span class="o">=</span> <span class="n">sock</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="n">fd_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span> <span class="o">=</span> <span class="n">sock</span>
<span class="n">ioloop</span> <span class="o">=</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">handle_client</span><span class="p">(</span><span class="n">cli_addr</span><span class="p">,</span> <span class="n">fd</span><span class="p">,</span> <span class="n">event</span><span class="p">):</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">fd_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">:</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">recv</span><span class="p">(</span><span class="mi">1024</span><span class="p">)</span>
<span class="k">if</span> <span class="n">data</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">" received '</span><span class="si">%s</span><span class="s2">' from </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cli_addr</span><span class="p">)</span>
<span class="c1"># 接收到消息更改事件为写, 用于发送数据到对端</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">update_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">WRITE</span><span class="p">)</span>
<span class="n">message_queue_map</span><span class="p">[</span><span class="n">s</span><span class="p">]</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">" closing </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">cli_addr</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">remove_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">del</span> <span class="n">message_queue_map</span><span class="p">[</span><span class="n">s</span><span class="p">]</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">WRITE</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">next_msg</span> <span class="o">=</span> <span class="n">message_queue_map</span><span class="p">[</span><span class="n">s</span><span class="p">]</span><span class="o">.</span><span class="n">get_nowait</span><span class="p">()</span>
<span class="k">except</span> <span class="n">Queue</span><span class="o">.</span><span class="n">Empty</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"</span><span class="si">%s</span><span class="s2"> queue empty"</span> <span class="o">%</span> <span class="n">cli_addr</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">update_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s1">'sending "</span><span class="si">%s</span><span class="s1">" to </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">next_msg</span><span class="p">,</span> <span class="n">cli_addr</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">next_msg</span><span class="p">)</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">ERROR</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">" exception on </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">cli_addr</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">remove_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">del</span> <span class="n">message_queue_map</span><span class="p">[</span><span class="n">s</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">handle_server</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">event</span><span class="p">):</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">fd_map</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">&</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">:</span>
<span class="n">conn</span><span class="p">,</span> <span class="n">cli_addr</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span>
<span class="nb">print</span> <span class="s2">" connection </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">cli_addr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">conn</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">conn_fd</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="n">fd_map</span><span class="p">[</span><span class="n">conn_fd</span><span class="p">]</span> <span class="o">=</span> <span class="n">conn</span>
<span class="n">handle</span> <span class="o">=</span> <span class="n">partial</span><span class="p">(</span><span class="n">handle_client</span><span class="p">,</span> <span class="n">cli_addr</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="c1"># 将cli_addr作为第一个参数</span>
<span class="c1"># 将连接和handle注册为读事件加入到 tornado ioloop</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">add_handler</span><span class="p">(</span><span class="n">conn_fd</span><span class="p">,</span> <span class="n">handle</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">)</span>
<span class="n">message_queue_map</span><span class="p">[</span><span class="n">conn</span><span class="p">]</span> <span class="o">=</span> <span class="n">Queue</span><span class="o">.</span><span class="n">Queue</span><span class="p">()</span> <span class="c1"># 创建对应的消息队列</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">add_handler</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">handle_server</span><span class="p">,</span> <span class="n">IOLoop</span><span class="o">.</span><span class="n">READ</span><span class="p">)</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div>
<p>上面代码就建立了一个非阻塞的高效的异步的echo server</p>使用WebQQ协议桥接XMPP和QQ群2013-03-14T16:09:00+08:002013-03-14T16:09:00+08:00coldtag:www.linuxzen.com,2013-03-14:/shi-yong-webqqxie-yi-qiao-jie-xmpphe-qqqun.html<h2 id="_1">介绍</h2>
<p>无意中看见有人利用WebQQ协议开发出Linux下Pidgin的插件, 让Pidgin来收发QQ消息, 突然想将<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>和QQ群来桥接起来一定非常有趣,这样就可以通过gtalk收发QQ来的消息, 不过前期还是想将<code>clubot</code>和QQ群桥接起来.</p>
<h2 id="_2">实施</h2>
<p>想到了就开始弄呗 …</p><h2 id="_1">介绍</h2>
<p>无意中看见有人利用WebQQ协议开发出Linux下Pidgin的插件, 让Pidgin来收发QQ消息, 突然想将<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>和QQ群来桥接起来一定非常有趣,这样就可以通过gtalk收发QQ来的消息, 不过前期还是想将<code>clubot</code>和QQ群桥接起来.</p>
<h2 id="_2">实施</h2>
<p>想到了就开始弄呗, 于是上网找了写有关WebQQ的协议, 首先写出了一个根据<code>urllib2</code>的版本并使用线程同时跑WebQQ和xmpp, 源码可以查看:
<a href="https://github.com/coldnight/qxbot/tree/threading_version">thread_version</a></p>
<h2 id="_3">优化</h2>
<p>上面的线程版效率不是很高, 由于都是网络请求, 所以想加入可以加入到pyxmpp2的mainloop中, 使用复用I/O模型来提高效率, 首先需要解决的是将http请求通过urllib2改为socket, 于是写出HTTPSock类来实现这个需求:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : Wood.D</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 13/03/04 09:58:26</span>
<span class="c1"># Desc : Http Socket 实现</span>
<span class="c1">#</span>
<span class="kn">import</span> <span class="nn">ssl</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">urllib</span>
<span class="kn">import</span> <span class="nn">urllib2</span>
<span class="kn">import</span> <span class="nn">httplib</span>
<span class="kn">import</span> <span class="nn">urlparse</span>
<span class="kn">import</span> <span class="nn">tempfile</span>
<span class="kn">import</span> <span class="nn">cookielib</span>
<span class="kn">from</span> <span class="nn">lib.utils</span> <span class="kn">import</span> <span class="n">Form</span>
<span class="k">class</span> <span class="nc">HTTPSock</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 构建支持Cookie的HTTP socket</span>
<span class="sd"> 供可复用的I/O模型调用"""</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">cookiefile</span> <span class="o">=</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">mktemp</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cookiejar</span> <span class="o">=</span> <span class="n">cookielib</span><span class="o">.</span><span class="n">MozillaCookieJar</span><span class="p">(</span><span class="n">cookiefile</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">make_request</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">form</span><span class="p">,</span> <span class="n">method</span> <span class="o">=</span> <span class="s2">"GET"</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 根据url 参数 构建 urllib2.Request """</span>
<span class="n">request</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">form</span><span class="p">,</span> <span class="n">Form</span><span class="p">):</span>
<span class="n">request</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"Content-Type"</span><span class="p">,</span> <span class="n">form</span><span class="o">.</span><span class="n">get_content_type</span><span class="p">())</span>
<span class="n">request</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"Content-Length"</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">form</span><span class="p">)))</span>
<span class="n">request</span><span class="o">.</span><span class="n">add_data</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">form</span><span class="p">))</span>
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">form</span><span class="p">,</span> <span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="nb">list</span><span class="p">,</span> <span class="nb">tuple</span><span class="p">)):</span>
<span class="n">params</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">form</span><span class="p">)</span>
<span class="k">if</span> <span class="n">method</span> <span class="o">==</span> <span class="s2">"GET"</span><span class="p">:</span>
<span class="n">url</span> <span class="o">=</span> <span class="s2">"</span><span class="si">{0}</span><span class="s2">?</span><span class="si">{1}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="n">request</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">request</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="n">request</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"Content-Type"</span><span class="p">,</span> <span class="s2">"application/x-www-form-urlencoded"</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cookiejar</span><span class="o">.</span><span class="n">add_cookie_header</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>
<span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">unredirected_hdrs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">request</span>
<span class="k">def</span> <span class="nf">make_response</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sock</span><span class="p">,</span> <span class="n">req</span><span class="p">,</span> <span class="n">method</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 根据socket和urlib2.Request 构建Response """</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">httplib</span><span class="o">.</span><span class="n">HTTPResponse</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">strict</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="n">method</span> <span class="o">=</span> <span class="n">method</span><span class="p">,</span> <span class="n">buffering</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">r</span><span class="o">.</span><span class="n">begin</span><span class="p">()</span>
<span class="n">r</span><span class="o">.</span><span class="n">recv</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">read</span>
<span class="n">fp</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">_fileobject</span><span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="n">close</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">resp</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">addinfourl</span><span class="p">(</span><span class="n">fp</span><span class="p">,</span> <span class="n">r</span><span class="o">.</span><span class="n">msg</span><span class="p">,</span> <span class="n">req</span><span class="o">.</span><span class="n">get_full_url</span><span class="p">())</span>
<span class="n">resp</span><span class="o">.</span><span class="n">code</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">status</span>
<span class="n">resp</span><span class="o">.</span><span class="n">msg</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">reason</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cookiejar</span><span class="o">.</span><span class="n">extract_cookies</span><span class="p">(</span><span class="n">resp</span><span class="p">,</span> <span class="n">req</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cookiejar</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="k">return</span> <span class="n">resp</span>
<span class="k">def</span> <span class="nf">make_http_sock_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 根据urllib2.Request 构建socket和用于发送的HTTP源数据 """</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_full_url</span><span class="p">()</span>
<span class="n">headers</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_data</span><span class="p">()</span>
<span class="n">parse</span> <span class="o">=</span> <span class="n">urlparse</span><span class="o">.</span><span class="n">urlparse</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">host</span><span class="p">,</span> <span class="n">port</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">splitport</span><span class="p">(</span><span class="n">parse</span><span class="o">.</span><span class="n">netloc</span><span class="p">)</span>
<span class="n">typ</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">scheme</span>
<span class="n">port</span> <span class="o">=</span> <span class="n">port</span> <span class="k">if</span> <span class="n">port</span> <span class="k">else</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">httplib</span><span class="p">,</span> <span class="n">typ</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"_PORT"</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_http_source</span><span class="p">(</span><span class="n">parse</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">headers</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="s2">"do_"</span> <span class="o">+</span> <span class="n">typ</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="s2">"do_"</span><span class="o">+</span><span class="n">typ</span><span class="p">)(</span><span class="n">host</span><span class="p">,</span> <span class="n">port</span><span class="p">),</span> <span class="n">data</span>
<span class="k">def</span> <span class="nf">do_http</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">host</span><span class="p">,</span> <span class="n">port</span><span class="p">):</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="o">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="o">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">settimeout</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="n">host</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">port</span><span class="p">)))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="k">return</span> <span class="n">sock</span>
<span class="k">def</span> <span class="nf">do_https</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">host</span><span class="p">,</span> <span class="n">port</span><span class="p">,</span> <span class="n">keyfile</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">certfile</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="o">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="o">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">settimeout</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="n">host</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">port</span><span class="p">)))</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">ssl</span><span class="o">.</span><span class="n">wrap_socket</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="n">keyfile</span><span class="p">,</span> <span class="n">certfile</span><span class="p">)</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="k">return</span> <span class="n">sock</span>
<span class="k">def</span> <span class="nf">get_http_source</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parse</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">headers</span><span class="p">):</span>
<span class="n">path</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">path</span>
<span class="n">query</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">query</span>
<span class="n">path</span> <span class="o">=</span> <span class="n">path</span> <span class="o">+</span> <span class="s2">"?"</span> <span class="o">+</span> <span class="n">query</span> <span class="k">if</span> <span class="n">query</span> <span class="k">else</span> <span class="n">path</span>
<span class="n">path</span> <span class="o">=</span> <span class="n">path</span> <span class="k">if</span> <span class="n">path</span> <span class="k">else</span> <span class="s2">"/"</span>
<span class="n">method</span> <span class="o">=</span> <span class="s2">"POST"</span> <span class="k">if</span> <span class="n">data</span> <span class="k">else</span> <span class="s2">"GET"</span>
<span class="n">_buffer</span><span class="o">=</span> <span class="p">[</span><span class="s2">"</span><span class="si">{0}</span><span class="s2"> </span><span class="si">{1}</span><span class="s2"> HTTP/1.1"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">method</span><span class="p">,</span> <span class="n">path</span><span class="p">)]</span>
<span class="n">e_headers</span> <span class="o">=</span> <span class="p">[(</span><span class="n">k</span><span class="o">.</span><span class="n">lower</span><span class="p">(),</span> <span class="n">v</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">headers</span><span class="o">.</span><span class="n">items</span><span class="p">()]</span>
<span class="n">headers</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Host"</span><span class="p">,</span> <span class="n">parse</span><span class="o">.</span><span class="n">netloc</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Connection"</span><span class="p">,</span> <span class="s2">"keep-alive"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Accept"</span><span class="p">,</span> <span class="s2">"*/*"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Accept-Charset"</span><span class="p">,</span> <span class="s2">"UTF-8,*;q=0.5"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Accept-Encoding"</span><span class="p">,</span> <span class="s2">"gzip,deflate,sdch"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Accept-Language"</span><span class="p">,</span> <span class="s2">"zh-CN,zh;q=0.8"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"User-Agent"</span><span class="p">,</span> <span class="s2">"Mozilla/5.0 (X11; Linux x86_64)"</span>\
<span class="s2">" AppleWebKit/537.11 (KHTML, like Gecko)"</span>\
<span class="s2">" Chrome/23.0.1271.97 Safari/537.11"</span><span class="p">))</span>
<span class="n">headers</span><span class="o">+=</span> <span class="n">e_headers</span>
<span class="k">if</span> <span class="n">data</span><span class="p">:</span>
<span class="n">headers</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="s2">"Content-Length"</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">headers</span><span class="p">:</span>
<span class="n">_buffer</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"</span><span class="si">{0}</span><span class="s2">: </span><span class="si">{1}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">key</span><span class="o">.</span><span class="n">title</span><span class="p">(),</span> <span class="n">value</span><span class="p">))</span>
<span class="n">_buffer</span><span class="o">.</span><span class="n">extend</span><span class="p">((</span><span class="s2">""</span><span class="p">,</span> <span class="s2">""</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="s2">"</span><span class="se">\r\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">_buffer</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
<span class="n">result</span> <span class="o">+=</span> <span class="n">data</span>
<span class="k">return</span> <span class="n">result</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">cookie</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">cookiejar</span><span class="o">.</span><span class="n">_cookies</span>
</code></pre></div>
<p>主要是根据<code>urllib2.Request</code>构建socket和socket要发送的数据, 然后将socket返回的数据构建成<code>response</code>, 然后编写一些handlers来加入到mainloop中去,优化后的版本:
<a href="https://github.com/coldnight/qxbot/tree/master">epoll_version</a></p>
<p>这个版本使用了epoll作为主循环, 更加高效.</p>
<h2 id="_4">最新版本</h2>
<p>最新版本分离了WebQQ作为一个包, 如仅需WebQQ的功能可以很方便的分离出来(当然要仿照pyxmpp2来实现一套事件机制), 源码:
<a href="https://github.com/coldnight/qxbot/">last</a></p>使用Pelican博客静态生成系统2013-03-14T15:12:00+08:002013-03-14T15:12:00+08:00coldtag:www.linuxzen.com,2013-03-14:/shi-yong-pelicanbo-ke-jing-tai-sheng-cheng-xi-tong.html<h1 id="_1">介绍</h1>
<p>最近流行使用静态生成的博客, 看了很多感觉很棒总想尝试一下, 大名鼎鼎的<code>Jekyll</code>使用ruby,看了很多文档介绍,依旧不能 …</p><h1 id="_1">介绍</h1>
<p>最近流行使用静态生成的博客, 看了很多感觉很棒总想尝试一下, 大名鼎鼎的<code>Jekyll</code>使用ruby,看了很多文档介绍,依旧不能入门使用,也许是用Ruby编写,对它比较迟钝.看到了一款Python编写的静态生成系统<code>Pelican</code>, 试用后除了模板较少外感觉还是不错的,使用<code>Markdown</code>和<code>Rst</code>编写.</p>
<h1 id="_2">安装</h1>
<p>可以试用<code>pip</code>来安装<code>Pelican</code>, 通过<code>pelican-quickstart</code>脚本可以快速构建一个站点,详细不在描述.可以看看它的文档</p>
<h2 id="_3">模板</h2>
<p>模板采用<code>neat</code>,这个模板设计简单(看我的页面就知道有多简单了)我非常喜欢, 但是有几个链接<code>bug</code>,bug不是很难找,很轻松就能找出来, 这里不列出来, 基于这个模板我做了一点更改.</p>
<h1 id="_4">使用</h1>
<p>在执行<code>pelican-quickstart</code>是开启Makefile, 可以通过make很简单的生成html.</p>
<div class="highlight"><pre><span></span><code>make<span class="w"> </span>html<span class="w"> </span><span class="c1"># 生成html</span>
make<span class="w"> </span>clean<span class="w"> </span><span class="c1"># 删除output</span>
</code></pre></div>
<h2 id="_5">技巧</h2>
<p>我是将生成的html通过nginx来打开(Nginx配置就很简单了, 这里就不做介绍), 那么如何确保更新呢?我将整个博客上传到github, 这样随时随地可以<code>clone</code>下来进行更改然后<code>push</code>上去,在服务器我通过crontab来定时更新html</p>
<div class="highlight"><pre><span></span><code>crontab<span class="w"> </span>-e
</code></pre></div>
<p>添加如下内容</p>
<div class="highlight"><pre><span></span><code>*/5<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span><span class="nb">cd</span><span class="w"> </span>/path/to/your/blogsource/<span class="w"> </span><span class="o">&&</span><span class="w"> </span>git<span class="w"> </span>pull<span class="w"> </span><span class="p">&</span>><span class="w"> </span>/dev/null
</code></pre></div>
<p>这样就可以没5分钟更新一次</p>vLog 一个使用Python编写的轻量级博客系统2013-02-05T16:43:00+08:002013-02-05T16:43:00+08:00coldtag:www.linuxzen.com,2013-02-05:/vlog-yi-ge-shi-yong-pythonbian-xie-de-qing-liang-ji-bo-ke-xi-tong.html<h2 id="_1">介绍</h2>
<h3 id="vlog">何为vLog</h3>
<p>大家有人可能注意到博客改变了,是的前面也有文章提到从wordpress迁移到vlog,但是何为vLog这里给大家简要的说明一下,vLog是我使用<code>Python</code>的<code>tornado</code>框架和<code>Jinja2</code>模板引擎,基于 …</p><h2 id="_1">介绍</h2>
<h3 id="vlog">何为vLog</h3>
<p>大家有人可能注意到博客改变了,是的前面也有文章提到从wordpress迁移到vlog,但是何为vLog这里给大家简要的说明一下,vLog是我使用<code>Python</code>的<code>tornado</code>框架和<code>Jinja2</code>模板引擎,基于MySQL数据的一个轻量级的博客系统,此系统功能比较薄弱,处于开发初期,使用<code>Markdown</code>的格式来抒写博文.</p>
<h3 id="vlog_1">为什么vLog</h3>
<p>vLog后台十分简单(可以说是简陋),功能也简单,就是一款简单的博客系统,提供了Python终端脚本,可以在终端来抒写博文, vLog使用一套非常简单的缓存系统,缓存使用memcached使得页面加载速度非常快.</p>
<h3 id="vlog_2">为什么不vLog</h3>
<p>相对与wordpress vlog非常简陋,仅仅提供简单的博客功能,而且使用Python编写主机方面支援不太多,虽然有<code>SAE</code>和<code>GAE</code>的支援,但是我没弄过,所以没有支援<code>SAE</code>和<code>GAE</code>(如果你有兴趣,可以添加相关支持)</p>
<h2 id="_2">安装</h2>
<h3 id="_3">平台</h3>
<ul>
<li>Linux</li>
<li>python2.7</li>
<li>MySQL</li>
<li>Memcached 1.4.5</li>
</ul>
<h3 id="_4">依赖库</h3>
<ul>
<li>tornado</li>
<li>jinja2</li>
<li>MySQLdb</li>
<li>pylibmc</li>
</ul>
<h3 id="_5">开始安装</h3>
<p>首先确认config.py的DEBUG是打开的,然后执行run.py,打开浏览器输入当前地址,会跳转到安装页面.按照提示安装,安装完毕后可以关闭DEBUG</p>
<h2 id="wordpress">从Wordpress中导入</h2>
<h3 id="wordpress_1">从Wordpress导出</h3>
<p>在wordpress管理后台选择工具->导出,下载导出文件可以导出一份xml</p>
<h3 id="_6">移动媒体文件</h3>
<p>将/path/to/your/wordpress/wp-content/uploads/下的所有文件移动到/path/to/your/vlog/web/static/upload 下即可</p>
<p>务必要先执行这一步然后再在后台里导入xml</p>
<h3 id="vlog_3">导入到vLog</h3>
<p>进入vLog后台,选择导入,浏览选中导出的xml, 然后选择开始,等待提示成功后即导入成功</p>
<h3 id="_7">手动更改没有生效的链接</h3>
<p>虽然我已经竭尽所能的让你手头的工作更少,但是还不够,还是存在许多需要手动更改的地方,
比如每篇文章的没有替换掉的图片链接</p>
<h3 id="_8">不足</h3>
<p>网站迁移后我已经尽力的来保持原来的链接有效,但是我仅仅知道我原来的wordpress的链接,所以仅仅兼容了我原来使用wordpress的旧链接,如果没能兼容您的wordpress的链接在此表示歉意,您可以自己添加提交给我,或者将您的链接提交给我由我来给您添加</p>
<h2 id="nginx">结合nginx</h2>
<p>参阅<a href="http://www.tornadoweb.cn/documentation#_14">tornado文档</a></p>
<h2 id="_9">源代码</h2>
<p>代码放在github上: <a href="https://github.com/coldnight/vlog">vLog</a></p>使用更加高效的epoll作为pyxmpp2的主循环2013-02-05T16:06:00+08:002013-02-05T16:06:00+08:00coldtag:www.linuxzen.com,2013-02-05:/shi-yong-geng-jia-gao-xiao-de-epollzuo-wei-pyxmpp2de-zhu-xun-huan.html<h2 id="_1">引子</h2>
<p>之前<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>使用的pyxmpp2的默认mainloop也就是一个poll的主循环,但是<code>clubot</code>上线后资源占用非常厉害,使用<code>strace</code>跟踪发现<code>clubot</code>在不停的<code>poll</code>,查看<code>pyxmpp2</code>代码发现<code>pyxmpp2</code>的<code>poll</code>在使用超 …</p><h2 id="_1">引子</h2>
<p>之前<a href="/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html">clubot</a>使用的pyxmpp2的默认mainloop也就是一个poll的主循环,但是<code>clubot</code>上线后资源占用非常厉害,使用<code>strace</code>跟踪发现<code>clubot</code>在不停的<code>poll</code>,查看<code>pyxmpp2</code>代码发现<code>pyxmpp2</code>的<code>poll</code>在使用超时阻塞时使用<code>最小</code>超时时间,而<code>最小</code>超时时间一直是0,所以会变成一个没有超时的非阻塞<code>poll</code>很浪费资源,不打算更改库代码,所以自己仿照poll的mainloop写了一个更加高效的<code>epoll</code>的mainloop</p>
<h2 id="_2">实现</h2>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 13/01/06 10:41:31</span>
<span class="c1"># Desc : Clubot epoll mainloop</span>
<span class="c1">#</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">absolute_import</span><span class="p">,</span> <span class="n">division</span>
<span class="kn">import</span> <span class="nn">select</span>
<span class="kn">from</span> <span class="nn">pyxmpp2.mainloop.interfaces</span> <span class="kn">import</span> <span class="n">HandlerReady</span><span class="p">,</span> <span class="n">PrepareAgain</span>
<span class="kn">from</span> <span class="nn">pyxmpp2.mainloop.base</span> <span class="kn">import</span> <span class="n">MainLoopBase</span>
<span class="kn">from</span> <span class="nn">plugin.util</span> <span class="kn">import</span> <span class="n">get_logger</span>
<span class="k">class</span> <span class="nc">EpollMainLoop</span><span class="p">(</span><span class="n">MainLoopBase</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" Main event loop based on the epoll() syscall on Linux system """</span>
<span class="n">READ_ONLY</span> <span class="o">=</span> <span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLIN</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLPRI</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLHUP</span> <span class="o">|</span>
<span class="n">select</span><span class="o">.</span><span class="n">EPOLLERR</span> <span class="o">|</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLET</span><span class="p">)</span>
<span class="n">READ_WRITE</span> <span class="o">=</span> <span class="n">READ_ONLY</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLOUT</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">settings</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">handlers</span><span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epoll</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">epoll</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span> <span class="o">=</span> <span class="p">{}</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span> <span class="o">=</span> <span class="p">{}</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_exists_fd</span> <span class="o">=</span> <span class="p">{}</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span> <span class="o">=</span> <span class="n">get_logger</span><span class="p">()</span>
<span class="n">MainLoopBase</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">settings</span><span class="p">,</span> <span class="n">handlers</span><span class="p">)</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">_add_io_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">handler</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_configure_io_handler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_configure_io_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">handler</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">check_events</span><span class="p">():</span>
<span class="k">return</span>
<span class="k">if</span> <span class="n">handler</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">:</span>
<span class="n">old_fileno</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span>
<span class="n">prepared</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_prepare_io_handler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">old_fileno</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">prepared</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">fileno</span> <span class="o">=</span> <span class="n">handler</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="k">if</span> <span class="n">old_fileno</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="n">fileno</span> <span class="o">!=</span> <span class="n">old_fileno</span><span class="p">:</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">old_fileno</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_exists</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">old_fileno</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epoll</span><span class="o">.</span><span class="n">unregister</span><span class="p">(</span><span class="n">old_fileno</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">prepared</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span> <span class="o">=</span> <span class="n">fileno</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">fileno</span><span class="p">:</span>
<span class="k">return</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fileno</span><span class="p">]</span> <span class="o">=</span> <span class="n">handler</span>
<span class="n">events</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">if</span> <span class="n">handler</span><span class="o">.</span><span class="n">is_readable</span><span class="p">():</span>
<span class="n">events</span> <span class="o">|=</span> <span class="bp">self</span><span class="o">.</span><span class="n">READ_ONLY</span>
<span class="k">if</span> <span class="n">handler</span><span class="o">.</span><span class="n">is_writable</span><span class="p">():</span>
<span class="n">events</span> <span class="o">|=</span> <span class="bp">self</span><span class="o">.</span><span class="n">READ_WRITE</span>
<span class="k">if</span> <span class="n">events</span><span class="p">:</span>
<span class="k">if</span> <span class="n">fileno</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_exists_fd</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epoll</span><span class="o">.</span><span class="n">modify</span><span class="p">(</span><span class="n">fileno</span><span class="p">,</span> <span class="n">events</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_exists_fd</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="n">fileno</span><span class="p">:</span><span class="mi">1</span><span class="p">})</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epoll</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">fileno</span><span class="p">,</span> <span class="n">events</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_prepare_io_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">handler</span><span class="p">):</span>
<span class="n">ret</span> <span class="o">=</span> <span class="n">handler</span><span class="o">.</span><span class="n">prepare</span><span class="p">()</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ret</span><span class="p">,</span> <span class="n">HandlerReady</span><span class="p">):</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span>
<span class="n">prepared</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ret</span><span class="p">,</span> <span class="n">PrepareAgain</span><span class="p">):</span>
<span class="k">if</span> <span class="n">ret</span><span class="o">.</span><span class="n">timeout</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span><span class="p">,</span> <span class="n">ret</span><span class="o">.</span><span class="n">timeout</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span> <span class="o">=</span> <span class="n">ret</span><span class="o">.</span><span class="n">timeout</span>
<span class="n">prepared</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">TypeError</span><span class="p">(</span><span class="s2">"Unexpected result from prepare()"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">prepared</span>
<span class="k">def</span> <span class="nf">_remove_io_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">handler</span><span class="p">):</span>
<span class="k">if</span> <span class="n">handler</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">:</span>
<span class="n">old_fileno</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">[</span><span class="n">handler</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">old_fileno</span> <span class="o">=</span> <span class="n">handler</span><span class="o">.</span><span class="n">fileno</span><span class="p">()</span>
<span class="k">if</span> <span class="n">old_fileno</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">old_fileno</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_exists</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">old_fileno</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epoll</span><span class="o">.</span><span class="n">unregister</span><span class="p">(</span><span class="n">old_fileno</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">loop_iteration</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">timeout</span> <span class="o">=</span> <span class="mi">60</span><span class="p">):</span>
<span class="n">next_timeout</span><span class="p">,</span> <span class="n">sources_handled</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_call_timeout_handlers</span><span class="p">()</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">check_events</span><span class="p">():</span>
<span class="k">return</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_quit</span><span class="p">:</span>
<span class="k">return</span> <span class="n">sources_handled</span>
<span class="k">for</span> <span class="n">handler</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_unprepared_handlers</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_configure_io_handler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">timeout</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="n">timeout</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_timeout</span><span class="p">)</span>
<span class="k">if</span> <span class="n">next_timeout</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">timeout</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="n">next_timeout</span><span class="p">,</span> <span class="n">timeout</span><span class="p">)</span>
<span class="k">if</span> <span class="n">timeout</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="n">timeout</span> <span class="o">+=</span> <span class="mi">1</span> <span class="c1"># 带有超时的非阻塞,解约资源</span>
<span class="n">events</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">epoll</span><span class="o">.</span><span class="n">poll</span><span class="p">(</span><span class="n">timeout</span><span class="p">)</span>
<span class="k">for</span> <span class="n">fd</span><span class="p">,</span> <span class="n">flag</span> <span class="ow">in</span> <span class="n">events</span><span class="p">:</span>
<span class="k">if</span> <span class="n">flag</span> <span class="o">&</span> <span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLIN</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLPRI</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLET</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span><span class="o">.</span><span class="n">handle_read</span><span class="p">()</span>
<span class="k">if</span> <span class="n">flag</span> <span class="o">&</span> <span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLOUT</span><span class="o">|</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLET</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span><span class="o">.</span><span class="n">handle_write</span><span class="p">()</span>
<span class="k">if</span> <span class="n">flag</span> <span class="o">&</span> <span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLERR</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLET</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span><span class="o">.</span><span class="n">handle_err</span><span class="p">()</span>
<span class="k">if</span> <span class="n">flag</span> <span class="o">&</span> <span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">EPOLLHUP</span> <span class="o">|</span> <span class="n">select</span><span class="o">.</span><span class="n">EPOLLET</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">]</span><span class="o">.</span><span class="n">handle_hup</span><span class="p">()</span>
<span class="c1">#if flag & select.EPOLLNVAL:</span>
<span class="c1">#self._handlers[fd].handle_nval()</span>
<span class="n">sources_handled</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_configure_io_handler</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_handlers</span><span class="p">[</span><span class="n">fd</span><span class="p">])</span>
<span class="k">return</span> <span class="n">sources_handled</span>
</code></pre></div>
<h2 id="_3">使用</h2>
<p>如何使用新的mainloop?只需在实例化Client时传入</p>
<div class="highlight"><pre><span></span><code><span class="n">mainloop</span> <span class="o">=</span> <span class="n">EpollMainLoop</span><span class="p">(</span><span class="n">settings</span><span class="p">)</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">Client</span><span class="p">(</span><span class="n">my_jid</span><span class="p">,</span> <span class="p">[</span><span class="bp">self</span><span class="p">,</span> <span class="n">version_provider</span><span class="p">],</span> <span class="n">settings</span><span class="p">,</span> <span class="n">mainloop</span><span class="p">)</span>
</code></pre></div>
<p>这样就会使用epoll作为mainloop</p>
<h2 id="_4">注意</h2>
<p>epoll仅仅在<code>Linux</code>下支持</p>
<h2 id="_5">写在后面的话</h2>
<p>欢迎大家加入到<code>clubot@vim-cn.com</code>中来讨论有关Python/vim/Linux/开源等话题</p>Python 重复安装包报错2013-01-25T14:55:00+08:002013-01-25T14:55:00+08:00coldtag:www.linuxzen.com,2013-01-25:/python-zhong-fu-an-zhuang-bao-bao-cuo.html<p>最近写程序用到argparse总是会报错</p>
<div class="highlight"><pre><span></span><code>/usr/lib/python2.7/dist-packages/pygments/plugin.py:39: UserWarning: Module argparse was already imported from /usr/lib/python2.7/argparse.pyc, but /usr/local/lib/python2.7/dist-packages is being added to sys.path
</code></pre></div>
<p>一开始也没怎么 …</p><p>最近写程序用到argparse总是会报错</p>
<div class="highlight"><pre><span></span><code>/usr/lib/python2.7/dist-packages/pygments/plugin.py:39: UserWarning: Module argparse was already imported from /usr/lib/python2.7/argparse.pyc, but /usr/local/lib/python2.7/dist-packages is being added to sys.path
</code></pre></div>
<p>一开始也没怎么在意,不影响什么,但是后来总是出来严重影响心情,google 了没啥明确的报错,后来仔细看报错信息,想起来以前可能庄过argparse这个包测试,但是没有删除
执行<code>ls /usr/local/lib/python2.7/dist-packages/</code>查看果然有一个argparse.</p>
<p>执行<code>sudo pip uninstall argparse</code>烦人的错误提示消失了.</p>Linux 下使用Python截图自动分享2013-01-22T16:59:00+08:002013-01-22T16:59:00+08:00coldtag:www.linuxzen.com,2013-01-22:/linux-xia-shi-yong-pythonjie-tu-zi-dong-fen-xiang.html<h2 id="_1">引子</h2>
<p>Linux下不支持QQ等功能丰富的IM,虽然可以通过wine运行QQ2012,但是还是喜欢在gtalk群中聊天,gtalk群不支持图片方式,这就要靠我们大家自己来解决了,<a href="http://eleveni386.7axu.com">eleven</a>开放了一个Image上传和显示接口,提供了使用<code>curl</code>来解决,但 …</p><h2 id="_1">引子</h2>
<p>Linux下不支持QQ等功能丰富的IM,虽然可以通过wine运行QQ2012,但是还是喜欢在gtalk群中聊天,gtalk群不支持图片方式,这就要靠我们大家自己来解决了,<a href="http://eleveni386.7axu.com">eleven</a>开放了一个Image上传和显示接口,提供了使用<code>curl</code>来解决,但是我们公司的网络使用<code>squid</code>禁止了<code>curl</code>的访问,所以整天看他们这么爽的分享图片我也不甘心阿,所以就使用Python写了一个分享图片的脚本</p>
<h2 id="_2">实现</h2>
<p>使用scrot截图,然后使用urllib2库上传图片,如果存在PyQt4库则会将结果放到剪贴板上,如果不存在则输出,自行复制</p>
<h2 id="_3">代码</h2>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 13/01/21 09:54:39</span>
<span class="c1"># Desc : 贴代码和图片</span>
<span class="c1">#</span>
<span class="kn">import</span> <span class="nn">urllib2</span><span class="o">,</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">mimetools</span>
<span class="kn">import</span> <span class="nn">mimetypes</span>
<span class="kn">import</span> <span class="nn">itertools</span>
<span class="n">__host__</span> <span class="o">=</span> <span class="s2">"http://eleveni386.7axu.com"</span>
<span class="k">class</span> <span class="nc">Form</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">form_fields</span> <span class="o">=</span> <span class="p">[]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">files</span> <span class="o">=</span> <span class="p">[]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">boundary</span> <span class="o">=</span> <span class="n">mimetools</span><span class="o">.</span><span class="n">choose_boundary</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">content_type</span> <span class="o">=</span> <span class="s2">"application/x-www-form-urlencoded"</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">get_content_type</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">content_type</span>
<span class="k">def</span> <span class="nf">add_field</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">form_fields</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">))</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">add_file</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fieldname</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="n">fileHandle</span><span class="p">,</span> <span class="n">mimetype</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">body</span> <span class="o">=</span> <span class="n">fileHandle</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">if</span> <span class="n">mimetype</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">mimetype</span> <span class="o">=</span> <span class="p">(</span> <span class="n">mimetypes</span><span class="o">.</span><span class="n">guess_type</span><span class="p">(</span><span class="n">filename</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="ow">or</span>
<span class="s1">'applicatioin/octet-stream'</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">files</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">fieldname</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="n">mimetype</span><span class="p">,</span> <span class="n">body</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">content_type</span> <span class="o">=</span> <span class="s1">'multipart/form-data; boundary=</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">boundary</span>
<span class="k">return</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">parts</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">part_boundary</span> <span class="o">=</span> <span class="s1">'--'</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">boundary</span>
<span class="n">parts</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span>
<span class="p">[</span> <span class="n">part_boundary</span><span class="p">,</span>
<span class="s1">'Content-Disposition: form-data; name="</span><span class="si">%s</span><span class="s1">"'</span> <span class="o">%</span> <span class="n">name</span><span class="p">,</span>
<span class="s1">''</span><span class="p">,</span>
<span class="n">value</span><span class="p">,</span>
<span class="p">]</span>
<span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">form_fields</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">files</span><span class="p">:</span>
<span class="n">parts</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span>
<span class="n">part_boundary</span><span class="p">,</span>
<span class="s1">'Content-Disposition: form-data; name="</span><span class="si">%s</span><span class="s1">"; filename="</span><span class="si">%s</span><span class="s1">"'</span> <span class="o">%</span>\
<span class="p">(</span><span class="n">field_name</span><span class="p">,</span> <span class="n">filename</span><span class="p">),</span>
<span class="s1">'Content-Type: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">content_type</span><span class="p">,</span>
<span class="s1">''</span><span class="p">,</span>
<span class="n">body</span><span class="p">,</span>
<span class="p">]</span> <span class="k">for</span> <span class="n">field_name</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="n">content_type</span><span class="p">,</span> <span class="n">body</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">files</span><span class="p">)</span>
<span class="n">flattened</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="o">*</span><span class="n">parts</span><span class="p">))</span>
<span class="n">flattened</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'--'</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">boundary</span> <span class="o">+</span> <span class="s1">'--'</span><span class="p">)</span>
<span class="n">flattened</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">''</span><span class="p">)</span>
<span class="k">return</span> <span class="s1">'</span><span class="se">\r\n</span><span class="s1">'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">flattened</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">HttpHelper</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">form</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">method</span> <span class="o">=</span> <span class="s1">'GET'</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_url</span> <span class="o">=</span> <span class="n">url</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_form</span> <span class="o">=</span> <span class="n">form</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_body</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">form</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_method</span> <span class="o">=</span> <span class="n">method</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_dst_url</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">if</span> <span class="n">url</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">make_request</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">make_request</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_url</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_url</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s1">'http://'</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://'</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">_url</span>
<span class="bp">self</span><span class="o">.</span><span class="n">request</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_form</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"Content-Type"</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_form</span><span class="o">.</span><span class="n">get_content_type</span><span class="p">())</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"Content-Length"</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_body</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">add_data</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_body</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">add_header</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">val</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">change</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">params</span> <span class="o">=</span> <span class="p">{},</span> <span class="n">method</span> <span class="o">=</span> <span class="s1">'GET'</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_url</span> <span class="o">=</span> <span class="n">url</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_params</span> <span class="o">=</span> <span class="n">params</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_method</span> <span class="o">=</span> <span class="n">method</span>
<span class="bp">self</span><span class="o">.</span><span class="n">make_request</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">open</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_dst_url</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">geturl</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">except</span><span class="p">:</span>
<span class="k">return</span> <span class="n">content</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">argparse</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="n">parser</span> <span class="o">=</span> <span class="n">argparse</span><span class="o">.</span><span class="n">ArgumentParser</span><span class="p">()</span>
<span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="n">dest</span><span class="o">=</span><span class="s2">"path"</span><span class="p">,</span> <span class="n">nargs</span><span class="o">=</span><span class="s2">"?"</span><span class="p">)</span>
<span class="n">args</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span>
<span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">path</span><span class="p">:</span>
<span class="n">path</span> <span class="o">=</span> <span class="n">args</span><span class="o">.</span><span class="n">path</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">path</span> <span class="o">=</span> <span class="sa">r</span><span class="s2">"/tmp/tmpscrot.png"</span>
<span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s2">"scrot -s </span><span class="si">{0}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">path</span><span class="p">))</span>
<span class="n">form</span> <span class="o">=</span> <span class="n">Form</span><span class="p">()</span>
<span class="n">filename</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="n">path</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="n">form</span><span class="o">.</span><span class="n">add_file</span><span class="p">(</span><span class="n">fieldname</span><span class="o">=</span><span class="s1">'mypic'</span><span class="p">,</span> <span class="n">filename</span><span class="o">=</span><span class="n">filename</span><span class="p">,</span>
<span class="n">fileHandle</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="n">path</span><span class="p">))</span>
<span class="n">http</span> <span class="o">=</span> <span class="n">HttpHelper</span><span class="p">(</span> <span class="n">__host__</span> <span class="o">+</span> <span class="s1">'/Image/'</span><span class="p">,</span> <span class="n">form</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">http</span><span class="o">.</span><span class="n">open</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">PyQt4.QtGui</span> <span class="kn">import</span> <span class="n">QApplication</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">([])</span>
<span class="n">cb</span> <span class="o">=</span> <span class="n">QApplication</span><span class="o">.</span><span class="n">clipboard</span><span class="p">()</span>
<span class="n">cb</span><span class="o">.</span><span class="n">setText</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">except</span><span class="p">:</span>
<span class="nb">print</span> <span class="n">url</span>
</code></pre></div>
<h2 id="_4">安装</h2>
<p>将上面代码保存一个文件,放在<code>PATH</code>路径里,赋予<code>执行权限</code>即可</p>
<h2 id="_5">使用</h2>
<p>默认的不跟图片地址则会截图,截图完毕后自动分享,如安装了PyQt4库则会将结果放到剪贴板,如没有则输出结果.如果脚本给了图片路径参数则上传给定路径的图片</p>vLog使用Tornado框架结合memcached缓存页面2013-01-16T16:14:00+08:002013-01-16T16:14:00+08:00coldtag:www.linuxzen.com,2013-01-16:/vlogshi-yong-tornadokuang-jia-jie-he-memcachedhuan-cun-ye-mian.html<h2 id="_1">原因</h2>
<p>Blog是一个更新并不很频繁的一套系统,但是每次刷新页面都要更新数据库反而很浪费资源,添加静态页面生成是一个解决办法,同时缓存是一个更好的 …</p><h2 id="_1">原因</h2>
<p>Blog是一个更新并不很频繁的一套系统,但是每次刷新页面都要更新数据库反而很浪费资源,添加静态页面生成是一个解决办法,同时缓存是一个更好的主意,可以结合Memcached添加少量的代码进行缓存,而且免去去了每次更新文章都要重新生成静态页面,特别当页面特别多时.</p>
<h2 id="_2">实现</h2>
<p>主要通过页面的uri进行缓存,结合tornado.web.RequestHandler的prepare和on_finish方法函数,
prepare 主要是请求前执行,on_finish()是请求结束之前执行.在渲染模板时缓存页面内容,然后在请求前检测是否有缓存,如果有直接输出缓存,结束请求,在POST提交之后清空所有缓存,重新生成缓存,从而保证内容实时性.由于登录用户和普通用户的页面不相同,所以不缓存登录用户页面(代码中没有体现,请自行实现).主要python代码(省略了模板渲染的代码):</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 13/01/14 09:57:31</span>
<span class="c1"># Desc : </span>
<span class="c1">#</span>
<span class="kn">import</span> <span class="nn">config</span>
<span class="kn">import</span> <span class="nn">pylibmc</span>
<span class="kn">from</span> <span class="nn">tornado.web</span> <span class="kn">import</span> <span class="n">RequestHandler</span>
<span class="c1">#### 省略Cache类定义 #####</span>
<span class="k">class</span> <span class="nc">Memcached</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="n">_mc</span> <span class="o">=</span> <span class="n">pylibmc</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">Client</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">CACHE_HOST</span><span class="p">,</span> <span class="n">binary</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="n">config</span><span class="o">.</span><span class="n">CACHED</span><span class="p">:</span>
<span class="k">return</span> <span class="n">Memcached</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">Cache</span><span class="p">()</span>
<span class="k">def</span> <span class="fm">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">exc_type</span><span class="p">,</span> <span class="n">exc_val</span><span class="p">,</span> <span class="n">exc_tb</span><span class="p">):</span>
<span class="k">pass</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">get_cache</span><span class="p">(</span><span class="bp">cls</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_mc</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">default</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
<span class="n">r</span> <span class="o">=</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_mc</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">r</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">default</span>
<span class="k">return</span> <span class="n">r</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">set</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">timeout</span> <span class="o">=</span> <span class="mi">0</span><span class="p">):</span>
<span class="n">timeout</span> <span class="o">=</span> <span class="n">timeout</span> <span class="k">if</span> <span class="n">timeout</span> <span class="k">else</span> <span class="n">config</span><span class="o">.</span><span class="n">CACHE_TIMEOUT</span>
<span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_mc</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">timeout</span><span class="p">)</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">delete</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_mc</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">flush</span><span class="p">(</span><span class="bp">cls</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_mc</span><span class="o">.</span><span class="n">flush_all</span><span class="p">()</span>
<span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Memcached</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__setattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Memcached</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">BaseHandler</span><span class="p">(</span><span class="n">RequestHandler</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 继承tornado请求基类,重写 prepare和on_finish方法 """</span>
<span class="n">cache</span> <span class="o">=</span> <span class="n">Memcached</span>
<span class="k">def</span> <span class="nf">render</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">template_path</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 渲染模板 """</span>
<span class="c1"># 省略渲染模板代码</span>
<span class="n">content</span> <span class="o">=</span> <span class="s1">''</span> <span class="c1"># 渲染模板后的内容</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">method</span> <span class="o">==</span> <span class="s2">"GET"</span> <span class="ow">and</span> <span class="n">CACHED</span> <span class="ow">and</span> \
<span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"/admin"</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cache</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">uri</span><span class="p">,</span> <span class="n">content</span><span class="p">)</span> <span class="c1"># 将渲染后的内容缓存起来</span>
<span class="bp">self</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">prepare</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">BaseHandler</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">prepare</span><span class="p">()</span>
<span class="c1"># 如果请求是GET方法,而且不是请求后台</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">method</span> <span class="o">==</span> <span class="s2">"GET"</span> <span class="ow">and</span> <span class="n">CACHED</span> <span class="ow">and</span> \
<span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"/admin"</span><span class="p">):</span>
<span class="c1"># 尝试获取当前页面的缓存</span>
<span class="n">cache</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">cache</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">uri</span><span class="p">)</span>
<span class="c1"># 获取缓存则输出页面,结束请求</span>
<span class="k">if</span> <span class="n">cache</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">finish</span><span class="p">(</span><span class="n">cache</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">on_finish</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 重写结束请求前的方法函数 """</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">method</span> <span class="o">==</span> <span class="s2">"POST"</span><span class="p">:</span>
<span class="c1"># 如果遇到POST提交则清空缓存</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cache</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
</code></pre></div>
<p>缓存系统在<code>redis</code>和<code>Memcached</code>选择了很久,因为只是单纯的缓存页面所以最后选择了<code>memcached</code>,使用<code>pylibmc</code> python库.</p>
<h2 id="_3">测试</h2>
<p>使用webbench 网站压力测试对比了缓存前后的结果:
使用缓存前</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>webbench<span class="w"> </span>-c<span class="w"> </span><span class="m">500</span><span class="w"> </span>-t<span class="w"> </span><span class="m">30</span><span class="w"> </span>http://www.linuxzen.com/
Webbench<span class="w"> </span>-<span class="w"> </span>Simple<span class="w"> </span>Web<span class="w"> </span>Benchmark<span class="w"> </span><span class="m">1</span>.5
Copyright<span class="w"> </span><span class="o">(</span>c<span class="o">)</span><span class="w"> </span>Radim<span class="w"> </span>Kolar<span class="w"> </span><span class="m">1997</span>-2004,<span class="w"> </span>GPL<span class="w"> </span>Open<span class="w"> </span>Source<span class="w"> </span>Software.
Benchmarking:<span class="w"> </span>GET<span class="w"> </span>http://www.linuxzen.com/
<span class="m">500</span><span class="w"> </span>clients,<span class="w"> </span>running<span class="w"> </span><span class="m">30</span><span class="w"> </span>sec.
<span class="nv">Speed</span><span class="o">=</span><span class="m">54</span><span class="w"> </span>pages/min,<span class="w"> </span><span class="m">38160</span><span class="w"> </span>bytes/sec.
Requests:<span class="w"> </span><span class="m">27</span><span class="w"> </span>susceed,<span class="w"> </span><span class="m">0</span><span class="w"> </span>failed.
</code></pre></div>
<p>使用缓存后:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>webbench<span class="w"> </span>-c<span class="w"> </span><span class="m">500</span><span class="w"> </span>-t<span class="w"> </span><span class="m">30</span><span class="w"> </span>http://www.linuxzen.com/
Webbench<span class="w"> </span>-<span class="w"> </span>Simple<span class="w"> </span>Web<span class="w"> </span>Benchmark<span class="w"> </span><span class="m">1</span>.5
Copyright<span class="w"> </span><span class="o">(</span>c<span class="o">)</span><span class="w"> </span>Radim<span class="w"> </span>Kolar<span class="w"> </span><span class="m">1997</span>-2004,<span class="w"> </span>GPL<span class="w"> </span>Open<span class="w"> </span>Source<span class="w"> </span>Software.
Benchmarking:<span class="w"> </span>GET<span class="w"> </span>http://www.linuxzen.com/
<span class="m">500</span><span class="w"> </span>clients,<span class="w"> </span>running<span class="w"> </span><span class="m">30</span><span class="w"> </span>sec.
<span class="nv">Speed</span><span class="o">=</span><span class="m">256</span><span class="w"> </span>pages/min,<span class="w"> </span><span class="m">238544</span><span class="w"> </span>bytes/sec.
Requests:<span class="w"> </span><span class="m">128</span><span class="w"> </span>susceed,<span class="w"> </span><span class="m">0</span><span class="w"> </span>failed.
</code></pre></div>
<p>明显快了很多...</p>Vim配置系列(二) —- 好看的statusline2013-01-05T15:01:00+08:002013-01-05T15:01:00+08:00coldtag:www.linuxzen.com,2013-01-05:/vimpei-zhi-xi-lie-er-hao-kan-de-statusline.html<p>Vim是一款文本编辑器,但是这并不影响它有一个好看的外观,大家都知道Vim可以通过配色方案来改变Vim的外观,满足一些‘好色之徒’,之前大家可能也主意到截图中一个非常漂亮 …</p><p>Vim是一款文本编辑器,但是这并不影响它有一个好看的外观,大家都知道Vim可以通过配色方案来改变Vim的外观,满足一些‘好色之徒’,之前大家可能也主意到截图中一个非常漂亮的statusline,这是通过Vim的一个Powerline的插件实现的.之前我们配置了Vundle的插件管理(传送门)我们可以用Vundle安装Powerline,在.vimrc(Windows可能是_vimrc)中添加:</p>
<div class="highlight"><pre><span></span><code>Bundle <span class="s2">"Lokaltog/vim-powerline"</span>
</code></pre></div>
<p>然后重新打开vim执行</p>
<div class="highlight"><pre><span></span><code><span class="p">:</span>BundleInstall
</code></pre></div>
<p>如果你和github畅通的话就会顺利安装插件,然后在.vimrc里添加</p>
<div class="highlight"><pre><span></span><code><span class="k">set</span> <span class="nb">laststatus</span><span class="p">=</span><span class="m">2</span>
<span class="k">let</span> <span class="k">g</span>:Powerline_symbols<span class="p">=</span><span class="s1">'unicode'</span>
</code></pre></div>
<p>如果gvim打开需要使用/path/to/your/bundle/vim-powerline/fontpatcher/fontpatcher给当前gvim使用的字体打上补丁(依赖需要fontforge和python)</p>
<p>然后重新打开vim你就会发现一个漂亮的statusline</p>
<p><img alt="好看的statusline" src="/upload/VimPythonComment1.png"></p>Vim 结合Python编写的翻译插件2013-01-04T17:58:00+08:002013-01-04T17:58:00+08:00coldtag:www.linuxzen.com,2013-01-04:/vim-jie-he-pythonbian-xie-de-fan-yi-cha-jian.html<p>最近重写了一下之前用Python写的一个终端翻译工具,想着Vim7.3 支持Python于是想将这个功能写为一个插件让Vim也支持翻译功能,现在英汉翻译比较完善, <Leader>t会翻译光标下单词,选中的翻译还不完善,仅仅是个半成品,在此抛砖引 …</leader></p><p>最近重写了一下之前用Python写的一个终端翻译工具,想着Vim7.3 支持Python于是想将这个功能写为一个插件让Vim也支持翻译功能,现在英汉翻译比较完善, <Leader>t会翻译光标下单词,选中的翻译还不完善,仅仅是个半成品,在此抛砖引玉.将下面代码复制保存为translate.vim 放到~/.vim/plugin目录下即可,代码如下:</p>
<div class="highlight"><pre><span></span><code><span class="c">" Author : cold</span>
<span class="c">" E-mail : wh_linux@126.com</span>
<span class="c">" Date : 2012/12/20 16:23</span>
<span class="c">" Desc : 英汉/汉英翻译插件</span>
<span class="c">" Useage :</span>
<span class="c">" <Leader> t 翻译当前光标下内容 //XXX 中文不行</span>
<span class="c">" <Leader> lt 翻译当前行</span>
<span class="c">" <Leader> vt 翻译选中的内容</span>
<span class="k">function</span> GetCursorWord<span class="p">()</span>
<span class="k">let</span> column <span class="p">=</span> get<span class="p">(</span>getpos<span class="p">(</span><span class="s1">'.'</span><span class="p">),</span> <span class="m">2</span><span class="p">,</span> <span class="m">0</span><span class="p">)</span> <span class="p">-</span> <span class="m">1</span>
<span class="k">let</span> line <span class="p">=</span> getline<span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="k">let</span> word <span class="p">=</span> strpart<span class="p">(</span>line<span class="p">,</span> column<span class="p">,</span> <span class="m">1</span><span class="p">)</span>
<span class="k">let</span> <span class="k">start</span> <span class="p">=</span> <span class="m">1</span>
<span class="k">while</span> <span class="m">1</span>
<span class="k">let</span> tmp <span class="p">=</span> strpart<span class="p">(</span>line<span class="p">,</span> column <span class="p">+</span> <span class="k">start</span><span class="p">,</span> <span class="m">1</span><span class="p">)</span>
<span class="k">if</span> tmp <span class="p">=~</span> <span class="s2">"[a-zA-Z]"</span>
<span class="k">let</span> word <span class="p">=</span> word . tmp
<span class="k">else</span>
<span class="k">break</span>
<span class="k">endif</span>
<span class="k">let</span> <span class="k">start</span> <span class="p">=</span> <span class="k">start</span> <span class="p">+</span> <span class="m">1</span>
<span class="k">endwhile</span>
<span class="k">let</span> <span class="k">start</span> <span class="p">=</span> <span class="m">1</span>
<span class="k">while</span> <span class="m">1</span>
<span class="k">let</span> tmp <span class="p">=</span> strpart<span class="p">(</span>line<span class="p">,</span> column <span class="p">-</span> <span class="k">start</span><span class="p">,</span> <span class="m">1</span><span class="p">)</span>
<span class="k">if</span> tmp <span class="p">=~</span> <span class="s1">'[a-zA-Z]'</span>
<span class="k">let</span> word <span class="p">=</span> tmp . word
<span class="k">else</span>
<span class="k">break</span>
<span class="k">endif</span>
<span class="k">let</span> <span class="k">start</span> <span class="p">=</span> <span class="k">start</span> <span class="p">+</span> <span class="m">1</span>
<span class="k">endwhile</span>
<span class="k">return</span> word
endfunc
<span class="k">function</span> Translate<span class="p">(</span><span class="k">m</span><span class="p">)</span>
<span class="c"> " m 1 -> 翻译当前行</span>
<span class="c"> " m 2 -> 翻译选中</span>
<span class="k">if</span> <span class="k">mode</span><span class="p">()</span> <span class="p">==</span> <span class="s1">'n'</span>
<span class="k">let</span> word <span class="p">=</span> GetCursorWord<span class="p">()</span> <span class="c">" 如果是命令模式则取当前字符下单词</span>
<span class="k">endif</span>
<span class="c"> "XXX Visual 模式mode()命令返回n</span>
<span class="k">if</span> <span class="k">a</span>:<span class="k">m</span> <span class="p">==</span> <span class="m">2</span>
<span class="k">let</span> word <span class="p">=</span> getreg<span class="p">(</span><span class="s1">'*'</span><span class="p">)</span> <span class="c">" 如果是选择模式获取选择块</span>
<span class="k">endif</span>
<span class="k">if</span> <span class="k">a</span>:<span class="k">m</span> <span class="p">==</span> <span class="m">1</span>
<span class="k">let</span> word <span class="p">=</span> getline<span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="k">endif</span>
<span class="k">python</span> <span class="o"><<</span> EOF
<span class="kn">import</span> <span class="nn">vim</span>
<span class="kn">import</span> <span class="nn">urllib</span>
<span class="kn">import</span> <span class="nn">urllib2</span>
<span class="k">class</span> <span class="nc">Translate</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 使用google进行英->汉, 汉->英的翻译 """</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">text</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="s1">'zh-CN'</span><span class="p">,</span> <span class="n">dst</span> <span class="o">=</span> <span class="s1">'en'</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s1">'http://translate.google.cn/translate_a/t'</span>
<span class="bp">self</span><span class="o">.</span><span class="n">params</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">client</span> <span class="o">=</span> <span class="s2">"t"</span><span class="p">,</span> <span class="n">text</span><span class="o">=</span><span class="n">text</span><span class="p">,</span>
<span class="n">hl</span> <span class="o">=</span> <span class="s1">'zh-CN'</span><span class="p">,</span> <span class="n">tl</span> <span class="o">=</span> <span class="n">dst</span><span class="p">,</span>
<span class="n">multires</span> <span class="o">=</span> <span class="s1">'1'</span><span class="p">,</span> <span class="n">prev</span> <span class="o">=</span> <span class="s1">'btn'</span><span class="p">,</span>
<span class="n">ssel</span> <span class="o">=</span> <span class="s1">'0'</span><span class="p">,</span> <span class="n">sc</span> <span class="o">=</span> <span class="s1">'1'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">src</span> <span class="o">!=</span> <span class="bp">self</span><span class="o">.</span><span class="n">params</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'hl'</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">params</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">sl</span> <span class="o">=</span> <span class="n">src</span><span class="p">)</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">loads</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 加载翻译结果 """</span>
<span class="k">while</span> <span class="s1">',,'</span> <span class="ow">in</span> <span class="n">content</span> <span class="ow">or</span> <span class="s1">'[,'</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">',,'</span><span class="p">,</span> <span class="s1">',"",'</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'[,'</span><span class="p">,</span> <span class="s1">'["",'</span><span class="p">)</span>
<span class="c1">#content = content.replace(',]', '"",]')</span>
<span class="n">content</span> <span class="o">=</span> <span class="nb">eval</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">content</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
<span class="n">desc</span> <span class="o">=</span> <span class="n">content</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">pinyin</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="k">if</span> <span class="n">result</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="k">else</span> <span class="n">result</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span>
<span class="n">others</span> <span class="o">=</span> <span class="s1">''</span>
<span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">desc</span><span class="p">:</span>
<span class="n">others</span> <span class="o">+=</span> <span class="n">d</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="s1">'</span><span class="se">\n</span><span class="s1">'</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">d</span><span class="p">[</span><span class="mi">1</span><span class="p">]:</span>
<span class="n">others</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\t</span><span class="si">{0}</span><span class="se">\t</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">d</span><span class="p">[</span><span class="mi">2</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">s</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="n">i</span><span class="p">:</span>
<span class="n">others</span> <span class="o">+=</span><span class="s1">','</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">s</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span> <span class="o">+</span> <span class="s1">'</span><span class="se">\n</span><span class="s1">'</span>
<span class="n">r</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">source</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">pinyin</span> <span class="o">=</span> <span class="n">pinyin</span><span class="p">,</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">r</span>
<span class="k">def</span> <span class="nf">translate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 调用google翻译 """</span>
<span class="n">params</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">params</span><span class="p">)</span>
<span class="n">req</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="n">req</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"User-Agent"</span><span class="p">,</span>
<span class="s2">"Mozilla/5.0+(compatible;+Googlebot/2.1;"</span>
<span class="s2">"++http://www.google.com/bot.html)"</span><span class="p">)</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">req</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">auto_translate</span><span class="p">(</span><span class="n">text</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" 自动检测当前语言进行翻译 """</span>
<span class="n">text</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">text</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">></span> <span class="sa">u</span><span class="s1">'z'</span><span class="p">:</span>
<span class="n">src</span> <span class="o">=</span> <span class="s1">'zh-CN'</span>
<span class="n">dst</span> <span class="o">=</span> <span class="s1">'en'</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">src</span> <span class="o">=</span> <span class="s1">'en'</span>
<span class="n">dst</span> <span class="o">=</span> <span class="s1">'zh-CN'</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">Translate</span><span class="p">(</span><span class="n">text</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">),</span> <span class="n">src</span><span class="p">,</span> <span class="n">dst</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">translate</span><span class="p">()</span>
<span class="k">return</span> <span class="n">result</span>
<span class="n">word</span> <span class="o">=</span> <span class="n">vim</span><span class="o">.</span><span class="n">eval</span><span class="p">(</span><span class="s1">'word'</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">auto_translate</span><span class="p">(</span><span class="n">word</span><span class="o">.</span><span class="n">strip</span><span class="p">())</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s1">'echo "源词: '</span> <span class="o">+</span> <span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"source"</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'"'</span><span class="p">)</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s1">'echo "结果: '</span> <span class="o">+</span> <span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"result"</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'"'</span><span class="p">)</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s1">'echo "拼音: '</span> <span class="o">+</span> <span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pinyin"</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'"'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'others'</span><span class="p">):</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s1">'echo "其他释义: "'</span><span class="p">)</span>
<span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s1">'echo "'</span> <span class="o">+</span> <span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'others'</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'"'</span><span class="p">)</span>
EOF
endfunc
nmap <span class="p"><</span>Leader<span class="p">></span><span class="k">t</span> :<span class="k">call</span> Translate<span class="p">(</span><span class="m">0</span><span class="p">)<</span><span class="k">cr</span><span class="p">></span>
nmap <span class="p"><</span>Leader<span class="p">></span><span class="k">lt</span> :<span class="k">call</span> Translate<span class="p">(</span><span class="m">1</span><span class="p">)<</span><span class="k">cr</span><span class="p">></span>
<span class="c">"XXX 无法判断当前模式,使用映射替代</span>
map <span class="p"><</span>Leader<span class="p">></span><span class="k">ts</span> :<span class="k">call</span> Translate<span class="p">(</span><span class="m">2</span><span class="p">)<</span><span class="k">cr</span><span class="p">></span>
</code></pre></div>Vim配置系列(一) ---- 插件管理2012-12-14T08:57:00+08:002012-12-14T08:57:00+08:00coldtag:www.linuxzen.com,2012-12-14:/vimpei-zhi-xi-lie-yi-cha-jian-guan-li.html<p>最近对Vim进行了一番较大的配置变动,所以就想写出一个系列来将配置过程分享下来,供需要的朋友参考.我们之前配置Vim插件是一大助力,可以帮助我们做一些比较 …</p><p>最近对Vim进行了一番较大的配置变动,所以就想写出一个系列来将配置过程分享下来,供需要的朋友参考.我们之前配置Vim插件是一大助力,可以帮助我们做一些比较cool或这比较实用的功能,但是我之前都是直接搜索插件然后下载下来,手动拷贝到相应的插件,这种感觉肯定是不爽,不管是Linux还是Python/Ruby都有一套自己的包管理器,可以比较智能的搜索/安装/升级/卸载包.Vim也有类似功能的插件Vundle,他是一款Vim插件管理器,依赖于git,git是一款非常棒的VCS这里不做介绍,有兴趣的可以了解一下.Vundle可以根据配置文件的github或其他git的路径自行安装/升级插件.下面我们来介绍如何安装:
首先在你的~/.vim下或者$VIM/vimfiles($VIM是vim的安装路径)创建bundle目录</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>~/.vim/bundle
</code></pre></div>
<p>然后使用git克隆Vundle项目:</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>https://github.com/gmarik/vundle.git<span class="w"> </span>~/.vim/bundle/vundle
</code></pre></div>
<p>然后在你的.vimrc里添加下面内容:</p>
<div class="highlight"><pre><span></span><code><span class="k">set</span> <span class="nb">nocompatible</span>
<span class="k">filetype</span> off <span class="c">" 先关闭文件类型</span>
<span class="k">set</span> <span class="nb">rtp</span><span class="p">+=~</span><span class="sr">/.vim/</span>bundle/vundle <span class="c">" 将vundle路径添加到插件vim路径</span>
<span class="k">call</span> vundle#rc<span class="p">()</span> <span class="c">" 执行Vundle初始化</span>
Bundle <span class="s1">'gmarik/vundle'</span> <span class="c">" 将Vundle加入到bundle</span>
<span class="k">filetype</span> indent plugin <span class="k">on</span> <span class="c">" 安装完后打开文件类型</span>
</code></pre></div>
<p>如需加上插件则在Bundle 'gmarik/vundle'后加上相应的Bundle,如果是github则可以只输入后面的相对路径,如果是其他git则需输入全部url,下面给出一个完整的例子:</p>
<div class="highlight"><pre><span></span><code><span class="k">set</span> <span class="nb">nocompatible</span>
<span class="k">filetype</span> off
<span class="k">set</span> <span class="nb">rtp</span><span class="p">+=~</span><span class="sr">/.vim/</span>bundle/vundle
<span class="k">call</span> vundle#rc<span class="p">()</span>
Bundle <span class="s1">'gmarik/vundle'</span>
<span class="c">"Bundle "MarcWeber/vim-addon-mw-utils"</span>
<span class="c">"Bundle "tomtom/tlib_vim"</span>
<span class="c">"Bundle "honza/snipmate-snippets"</span>
<span class="c">"Bundle "garbas/vim-snipmate"</span>
Bundle <span class="s2">"Shougo/neocomplcache"</span>
Bundle <span class="s2">"Lokaltog/vim-powerline"</span>
Bundle <span class="s2">"drakeguan/vim-vcscommand"</span>
Bundle <span class="s2">"scrooloose/nerdtree"</span>
Bundle <span class="s2">"pix/vim-taglist"</span>
Bundle <span class="s2">"nathanaelkane/vim-indent-guides"</span>
Bundle <span class="s2">"clones/vim-cecutil"</span>
Bundle <span class="s2">"tpope/vim-fugitive"</span>
<span class="c">"Bundle "c9s/bufexplorer"</span>
Bundle <span class="s2">"jnwhiteh/vim-golang"</span>
Bundle <span class="s2">"kevinw/pyflakes-vim"</span>
Bundle <span class="s2">"mbriggs/mark.vim"</span>
<span class="c">"Bundle "vim-scripts/TabBar"</span>
Bundle <span class="s2">"vim-scripts/DrawIt"</span>
Bundle <span class="s2">"vim-scripts/calendar.vim--Matsumoto"</span>
Bundle <span class="s2">"vim-scripts/Python-mode-klen"</span>
<span class="c">"Bundle "vim-scripts/pydoc.vim"</span>
Bundle <span class="s2">"vim-scripts/VOoM"</span>
Bundle <span class="s2">"vim-scripts/qiushibaike"</span>
Bundle <span class="s2">"vim-scripts/AuthorInfo"</span>
Bundle <span class="s2">"vim-scripts/javacomplete"</span>
Bundle <span class="s2">"vim-scripts/javaDoc.vim"</span>
Bundle <span class="s2">"drmingdrmer/xptemplate.git"</span>
Bundle <span class="s2">"vim-scripts/Java-Syntax-and-Folding"</span>
<span class="k">filetype</span> indent plugin <span class="k">on</span>
</code></pre></div>
<p>以上是我的Vundle配置,再后面的文章会对其中一些做出介绍:</p>
<p>配置完后保存使用Vim打开任意文档执行</p>
<div class="highlight"><pre><span></span><code><span class="p">:</span>BundleInstall
</code></pre></div>
<p>如果网络通畅Bundle会逐个安装每个插件.</p>
<p>同时还有:BundleSearch用于搜索插件,非常方便.</p>Vim打开Python源码自动添加#!行和编码行 升级版2012-12-13T14:31:00+08:002012-12-13T14:31:00+08:00coldtag:www.linuxzen.com,2012-12-13:/vimda-kai-pythonyuan-ma-zi-dong-tian-jia-xing-he-bian-ma-xing-sheng-ji-ban.html<p>之前给大家分享过一个打开Python源代码时自动添加#!行和编码行来避免一些重复的工作,那个是因为需要大量编写时临时的解决方案,后来使 …</p><p>之前给大家分享过一个打开Python源代码时自动添加#!行和编码行来避免一些重复的工作,那个是因为需要大量编写时临时的解决方案,后来使用中会出现一些问题,比如查看别人源码时也会更改一些东西,从而造成git不必要的更新和手动删除的额外动作,所以又写了一个,只是在文件是新打开文件时或者空文件才自动添加的方法,同时再打开python源文件将这个方法绑定要F4上可以手动添加,并会判断是否有这两行,如果有则不执行动作,同时也添加了一些辅助性注释, 比如 作者/邮箱/创建日期和描述,代码实现如下:</p>
<div class="highlight"><pre><span></span><code><span class="c">"Python 注释</span>
<span class="k">function</span> InsertPythonComment<span class="p">()</span>
exe <span class="s1">'normal'</span>.<span class="m">1</span>.<span class="s1">'G'</span>
<span class="k">let</span> line <span class="p">=</span> getline<span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="k">if</span> line <span class="p">=~</span> <span class="s1">'^#!.*$'</span> <span class="p">||</span> line <span class="p">=~</span> <span class="s1">'^#.*coding:.*$'</span>
<span class="k">return</span>
<span class="k">endif</span>
normal O
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'#!/usr/bin/env python'</span><span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'# -*- coding:utf-8 -*-'</span><span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'#'</span><span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'# Author : '</span>.<span class="k">g</span>:python_author<span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'# E-mail : '</span>.<span class="k">g</span>:python_email<span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'# Date : '</span>.strftime<span class="p">(</span><span class="s2">"%y/%m/%d %H:%M:%S"</span><span class="p">))</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'# Desc : '</span><span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> setline<span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'#'</span><span class="p">)</span>
normal <span class="k">o</span>
<span class="k">call</span> cursor<span class="p">(</span><span class="m">7</span><span class="p">,</span> <span class="m">17</span><span class="p">)</span>
<span class="k">endfunction</span>
<span class="k">function</span> InsertCommentWhenOpen<span class="p">()</span>
<span class="k">if</span> <span class="k">a</span>:lastline <span class="p">==</span> <span class="m">1</span> && <span class="p">!</span>getline<span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="k">call</span> InsertPythonComment<span class="p">()</span>
<span class="k">end</span>
endfunc
<span class="k">au</span> <span class="nb">FileType</span> python :%<span class="k">call</span> InsertCommentWhenOpen<span class="p">()</span>
<span class="k">au</span> <span class="nb">FileType</span> python map <span class="p"><</span>F4<span class="p">></span> :<span class="k">call</span> InsertPythonComment<span class="p">()<</span><span class="k">cr</span><span class="p">></span>
</code></pre></div>
<p>将上面的代码放到你的vimrc中,同时在vimrc添加:</p>
<div class="highlight"><pre><span></span><code><span class="k">let</span> <span class="k">g</span>:python_author <span class="p">=</span> <span class="s1">'cold'</span> # 姓名
<span class="k">let</span> <span class="k">g</span>:python_email <span class="p">=</span> <span class="s1">'wh_linux@126.com'</span> # 邮箱
</code></pre></div>
<p>这样在每次打开空的python源文件时就会添加这些注释信息,并可以在非空没有这些注释的情况下按F4添加,配置玩后,打开空的python源文件效果如下:
<img alt="好看的statusline" src="/upload/VimPythonComment1.png"></p>Awesome+tmux+gnomeDo打造高效Linux桌面环境2012-12-04T13:21:00+08:002012-12-04T13:21:00+08:00coldtag:www.linuxzen.com,2012-12-04:/awesometmuxgnomedoda-zao-gao-xiao-linuxzhuo-mian-huan-jing.html<h2 id="_1">引言</h2>
<p>近期一直在Linux下工作,使用Ubuntu 11.10,经过一段时间的使用和磨合,终于打造出一套适合自己的高效Linux桌面环境,之前也在博客中零散的 …</p><h2 id="_1">引言</h2>
<p>近期一直在Linux下工作,使用Ubuntu 11.10,经过一段时间的使用和磨合,终于打造出一套适合自己的高效Linux桌面环境,之前也在博客中零散的写了几篇文章分享,在此做一番总结.</p>
<p>首先先放出桌面截图
<img alt="Awesome 桌面截图" src="/upload/MyDesktop.png"></p>
<h2 id="awesome">Awesome</h2>
<p>使用Ubuntu 11.10不习惯默认搭载的Unity,Gnome 3也不尽人如意,也使用xfce/openbox,但使用都不是很好,没有Windows的体验好,然后接触了Awesome,Awesome是一款平铺式窗体管理器,Awesome会去除窗口的标题栏等.会使窗口尽量小的占用桌面空间,而且大部分窗口操作都可以通过键盘来进行操作,免除了各位身为键盘高手的码农们频繁拿鼠标的烦恼.</p>
<h3 id="_2">安装</h3>
<p>Awesome Ubuntu下安装十分简单:</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>awesome
</code></pre></div>
<h3 id="_3">配置</h3>
<h4 id="_4">拷贝配置文件</h4>
<p>Awesome 的配置文件使用lua脚本,所以如果你会lua配置起来会得心应手,我们先拷贝一个基础配置文件,然后在这个基础上进行更改:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>/etc/xdg/awesome/rc.lua<span class="w"> </span>~/.config/awesome<span class="w"> </span><span class="c1"># ~/.confg下如没有awesome则手动创建</span>
</code></pre></div>
<h4 id="_5">配置自动启程序</h4>
<p>使用awesome之后之前设置的自动启动就会失效,因为Awesome启动是通过配置文件控制的,在配置文件(~/.config/awesome/rc.lua)加上如下内容可以配置自启动程序:</p>
<div class="highlight"><pre><span></span><code><span class="n">autorun</span> <span class="o">=</span> <span class="kc">true</span>
<span class="n">autorunApps</span> <span class="o">=</span>
<span class="p">{</span>
<span class="s2">"pidgin"</span><span class="p">,</span>
<span class="s2">"fcitx"</span><span class="p">,</span>
<span class="s2">"dbus-launch gnome-do"</span><span class="p">,</span>
<span class="s2">"guake"</span><span class="p">,</span>
<span class="s2">"/opt/qq2012/wineapp/qq/qq.sh"</span><span class="p">,</span>
<span class="p">}</span>
<span class="kr">if</span> <span class="n">autorun</span> <span class="kr">then</span>
<span class="kr">for</span> <span class="n">app</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="o">#</span><span class="n">autorunApps</span> <span class="kr">do</span>
<span class="n">awful</span><span class="p">.</span><span class="n">util</span><span class="p">.</span><span class="n">spawn_with_shell</span><span class="p">(</span><span class="n">autorunApps</span><span class="p">[</span><span class="n">app</span><span class="p">])</span>
<span class="kr">end</span>
<span class="kr">end</span>
</code></pre></div>
<p>在autorunApps中添加要自动启动程序的命令即可在登录时启动相应的程序</p>
<h4 id="gnome">使用gnome桌面元素</h4>
<p>awesome仅仅是一款窗体管理器,如果不进行相应配置awesome会使用默认的x window元素,非常丑陋我们来配置awesome使用GNOME桌面管理:
在宿主目录创建.xinitrc,添加如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
gnome-settings-daemon<span class="w"> </span><span class="p">&</span><span class="w"> </span><span class="c1"># 使用gnome桌面元素</span>
nm-applet<span class="w"> </span>--sm-disable<span class="w"> </span><span class="p">&</span><span class="w"> </span><span class="c1"># 托盘栏添加网卡图标</span>
<span class="nb">exec</span><span class="w"> </span>awesome
</code></pre></div>
<p>如果不生效,则执行下面命令:</p>
<div class="highlight"><pre><span></span><code>ln<span class="w"> </span>-sf<span class="w"> </span>~/.xinitrc<span class="w"> </span>~/.xprofile
</code></pre></div>
<h3 id="_6">美化</h3>
<p>可以自己编写lua脚本对Awesome进行美化,当然还有很多已经写好的配置,我们叫他为主题
awesome的所有主题可以在 https://github.com/mikar/awesome-themes 上下载.
我们可以使用git将主题整体克隆下来</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git://github.com/mikar/awesome-themes.git<span class="w"> </span>~/.config/awesome/themes
</code></pre></div>
<p>然后修改~/.config/awesome/rc.lua的beautiful.init()的参数改为主题的路径:</p>
<div class="highlight"><pre><span></span><code><span class="n">beautiful</span><span class="p">.</span><span class="n">init</span><span class="p">(</span><span class="s2">"/your/home/path/.config/awesome/themes/bamboo/theme.lua"</span><span class="p">)</span>
</code></pre></div>
<p>如果背景图片无法显示,安装feh:</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>feh
</code></pre></div>
<h3 id="_7">使用</h3>
<p>配置完成后注销使用Awesome会话登录,Awesome使用Win键作为主键,几个常用的为:</p>
<div class="highlight"><pre><span></span><code>Win+num 可以在多个桌面切换
Win+Ctrl+r 可以重启Awesome,
Win+k/j可以切换窗口(类Vim操作),
Win+m/n可以最大/小化窗口,
Win+Shift+num 可以将当前窗口发送到其他桌面.
Win+Shift+C可以关闭当前窗口,
Win+Space可以切换布局,Awesome有多种布局,这里不作介绍,大家可以自己稍作尝试.
</code></pre></div>
<p>下面给出完整的快捷键:</p>
<div class="highlight"><pre><span></span><code>Mod4 + Enter 打开终端
Mod4 + r 执行程序或命令
Mod4 + w 打开Awesome主菜单(鼠标右键关闭)
Mod4 + Shift + c 关闭当前窗口或应用
Mod4 + Control + r 重启Awesome
Mod4 + Shift + q 退出Awesome
Mod4 + j 切换到下一个窗口
Mod4 + k 切换到前一个窗口
Mod4 + Left 查看前一个tag
Mod4 + Right 查看后一个tag
Mod4 + 1-9 切换到tag 1-9
Mod4 + Control + j 切换到下一个屏幕
Mod4 + Control + k 切换到上一个屏幕
Mod4 + Shift + j 当前窗口和前一个窗口互换位置
Mod4 + Shift + k 当前窗口和后一个窗口互换位置
Mod4 + h 把主区域的宽度增大5%
Mod4 + l 把主区域的宽度减少5%
Mod4 + m 最大化窗口
Mod4 + n 最小化窗口
Mod4 + Shift + h 增加主区域窗口的数量
Mod4 + Shift + l 减少主区域窗口的数量
Mod4 + Space 切换窗口布局
Mod4 + Shift + space 把当前tag更换为前一种布局
Mod4 + Control + space 切换当前窗口是否为浮动
Mod4 + Shift + i 显示当前窗口的Class和instance,这在写脚本的时候尤其有用
Mod4 + Shift + r 重绘当前窗口
Mod4 + t 标记窗口(可标记多个)
Mod4 + Shift + F1~F9 把标记的窗口移动到第1~9个标记上
Ctrl + Mod4 + 1~9 把当前桌面和1~9桌面是显示
Mod4 + 1~9 恢复
Mod4 + Esc 快速切换到上一个桌面
</code></pre></div>
<p>更加详尽的介绍和配置请参见http://josephpan.net/blog/?p=992</p>
<h2 id="tmux">Tmux</h2>
<p>tmux是一款替代screen的产品,除了拥有screen的基本功能外,还有窗口分隔,多人共享等功能.尤其是窗口分隔功能异常突出.同时tmux简化了很多快捷键.还支持vi/emacs风格的快捷键绑定</p>
<h3 id="_8">安装</h3>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>tmux
</code></pre></div>
<h3 id="_9">配置</h3>
<p>tmux 使用~/.tmux.conf作为配置文件,所以我们可以将配置添加到这个文件中,网上很多资料都将前缀键绑定到Ctrl+a,但我喜欢使用Ctrl+a跳到行首,所以我还是使用默认的前缀键Ctrl+b.下面是我的配置文件:</p>
<div class="highlight"><pre><span></span><code><span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">default</span><span class="o">-</span><span class="n">terminal</span><span class="w"> </span><span class="s2">"screen-256color"</span><span class="w"> </span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">display</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="mi">3000</span><span class="w"> </span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">escape</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="mi">0</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">history</span><span class="o">-</span><span class="n">limit</span><span class="w"> </span><span class="mi">65535</span><span class="w"> </span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">base</span><span class="o">-</span><span class="n">index</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">utf8</span><span class="w"> </span><span class="n">on</span><span class="w"> </span>
<span class="c1"># split window</span>
<span class="n">unbind</span><span class="w"> </span><span class="s1">'"'</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">splitw</span><span class="w"> </span><span class="o">-</span><span class="n">v</span><span class="w"> </span><span class="c1"># vertical split (prefix -)</span>
<span class="n">unbind</span><span class="w"> </span><span class="o">%</span>
<span class="n">bind</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">splitw</span><span class="w"> </span><span class="o">-</span><span class="n">h</span><span class="w"> </span><span class="c1"># horizontal split (prefix |)</span>
<span class="c1"># select pane</span>
<span class="n">bind</span><span class="w"> </span><span class="n">k</span><span class="w"> </span><span class="n">selectp</span><span class="w"> </span><span class="o">-</span><span class="n">U</span><span class="w"> </span><span class="c1"># above (prefix k)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">j</span><span class="w"> </span><span class="n">selectp</span><span class="w"> </span><span class="o">-</span><span class="n">D</span><span class="w"> </span><span class="c1"># below (prefix j)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">h</span><span class="w"> </span><span class="n">selectp</span><span class="w"> </span><span class="o">-</span><span class="n">L</span><span class="w"> </span><span class="c1"># left (prefix h)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="n">selectp</span><span class="w"> </span><span class="o">-</span><span class="n">R</span><span class="w"> </span><span class="c1"># right (prefix l)</span>
<span class="c1"># resize pane</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="w"> </span><span class="o">^</span><span class="n">k</span><span class="w"> </span><span class="n">resizep</span><span class="w"> </span><span class="o">-</span><span class="n">U</span><span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="c1"># upward (prefix Ctrl+k)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="w"> </span><span class="o">^</span><span class="n">j</span><span class="w"> </span><span class="n">resizep</span><span class="w"> </span><span class="o">-</span><span class="n">D</span><span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="c1"># downward (prefix Ctrl+j)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="w"> </span><span class="o">^</span><span class="n">h</span><span class="w"> </span><span class="n">resizep</span><span class="w"> </span><span class="o">-</span><span class="n">L</span><span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="c1"># to the left (prefix Ctrl+h)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="w"> </span><span class="o">^</span><span class="n">l</span><span class="w"> </span><span class="n">resizep</span><span class="w"> </span><span class="o">-</span><span class="n">R</span><span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="c1"># to the right (prefix Ctrl+l)</span>
<span class="c1"># swap pane</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">u</span><span class="w"> </span><span class="n">swapp</span><span class="w"> </span><span class="o">-</span><span class="n">U</span><span class="w"> </span><span class="c1"># swap with the previous pane (prefix Ctrl+u)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">d</span><span class="w"> </span><span class="n">swapp</span><span class="w"> </span><span class="o">-</span><span class="n">D</span><span class="w"> </span><span class="c1"># swap with the next pane (prefix Ctrl+d)</span>
<span class="c1"># misc</span>
<span class="n">bind</span><span class="w"> </span><span class="n">e</span><span class="w"> </span><span class="n">lastp</span><span class="w"> </span><span class="c1"># select the last pane (prefix e)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">e</span><span class="w"> </span><span class="n">last</span><span class="w"> </span><span class="c1"># select the last window (prefix Ctrl+e)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">q</span><span class="w"> </span><span class="n">killp</span><span class="w"> </span><span class="c1"># kill pane (prefix q)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">q</span><span class="w"> </span><span class="n">killw</span><span class="w"> </span><span class="c1"># kill window (prefix Ctrl+q)</span>
<span class="c1"># copy mode</span>
<span class="n">bind</span><span class="w"> </span><span class="n">Escape</span><span class="w"> </span><span class="n">copy</span><span class="o">-</span><span class="n">mode</span><span class="w"> </span><span class="c1"># enter copy mode (prefix Escape)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">p</span><span class="w"> </span><span class="n">pasteb</span><span class="w"> </span><span class="c1"># paste buffer (prefix Ctrl+p)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">t</span><span class="w"> </span><span class="n">vi</span><span class="o">-</span><span class="n">copy</span><span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="n">begin</span><span class="o">-</span><span class="n">selection</span><span class="w"> </span><span class="c1"># select (v)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">-</span><span class="n">t</span><span class="w"> </span><span class="n">vi</span><span class="o">-</span><span class="n">copy</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="n">copy</span><span class="o">-</span><span class="n">selection</span><span class="w"> </span><span class="c1"># copy (y)</span>
<span class="c1"># zoom pane <-> window </span>
<span class="c1"># see also: http://tmux.svn.sourceforge.net/viewvc/tmux/trunk/examples/tmux-zoom.sh</span>
<span class="n">bind</span><span class="w"> </span><span class="o">^</span><span class="n">z</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="s2">"tmux-zoom"</span>
<span class="c1"># app</span>
<span class="n">bind</span><span class="w"> </span><span class="o">!</span><span class="w"> </span><span class="n">splitw</span><span class="w"> </span><span class="n">htop</span><span class="w"> </span><span class="c1"># htop (prefix !)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">m</span><span class="w"> </span><span class="n">command</span><span class="o">-</span><span class="n">prompt</span><span class="w"> </span><span class="s2">"splitw 'exec man </span><span class="si">%%</span><span class="s2">'"</span><span class="w"> </span><span class="c1"># man (prefix m)</span>
<span class="n">bind</span><span class="w"> </span><span class="err">@</span><span class="w"> </span><span class="n">command</span><span class="o">-</span><span class="n">prompt</span><span class="w"> </span><span class="s2">"splitw 'exec perldoc -t -f </span><span class="si">%%</span><span class="s2">'"</span><span class="w"> </span><span class="c1"># perl func (prefix @)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">command</span><span class="o">-</span><span class="n">prompt</span><span class="w"> </span><span class="s2">"splitw 'exec perldoc -t -v </span><span class="si">%%</span><span class="s2">'"</span><span class="w"> </span><span class="c1"># perl var (prefix *)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">%</span><span class="w"> </span><span class="n">command</span><span class="o">-</span><span class="n">prompt</span><span class="w"> </span><span class="s2">"splitw 'exec perldoc -t </span><span class="si">%%</span><span class="s2">'"</span><span class="w"> </span><span class="c1"># perl doc (prefix %)</span>
<span class="n">bind</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">command</span><span class="o">-</span><span class="n">prompt</span><span class="w"> </span><span class="s2">"splitw 'exec ri </span><span class="si">%%</span><span class="s2">'"</span><span class="w"> </span><span class="c1"># ruby doc (prefix /)</span>
<span class="c1"># reload config (prefix r)</span>
<span class="n">bind</span><span class="w"> </span><span class="n">r</span><span class="w"> </span><span class="n">source</span><span class="w"> </span><span class="o">~/.</span><span class="n">tmux</span><span class="o">.</span><span class="n">conf</span><span class="w"> </span>\<span class="p">;</span><span class="w"> </span><span class="n">display</span><span class="w"> </span><span class="s2">"Configuration reloaded!"</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">mode</span><span class="o">-</span><span class="n">keys</span><span class="w"> </span><span class="n">vi</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">automatic</span><span class="o">-</span><span class="n">rename</span><span class="w"> </span><span class="n">off</span>
<span class="c1">#-- colorscheme --#</span>
<span class="c1"># see also: https://github.com/daethorian/conf-tmux/blob/master/colors/zenburn.conf</span>
<span class="c1"># modes</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">clock</span><span class="o">-</span><span class="n">mode</span><span class="o">-</span><span class="n">colour</span><span class="w"> </span><span class="n">colour223</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">mode</span><span class="o">-</span><span class="n">attr</span><span class="w"> </span><span class="n">bold</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">mode</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">colour223</span>
<span class="n">setw</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">mode</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">colour235</span>
<span class="c1"># panes</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">pane</span><span class="o">-</span><span class="n">border</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">default</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">pane</span><span class="o">-</span><span class="n">border</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">colour234</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">pane</span><span class="o">-</span><span class="n">active</span><span class="o">-</span><span class="n">border</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">default</span><span class="w"> </span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">pane</span><span class="o">-</span><span class="n">active</span><span class="o">-</span><span class="n">border</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">green</span>
<span class="c1"># messages</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">message</span><span class="o">-</span><span class="n">attr</span><span class="w"> </span><span class="n">bold</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">message</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">colour223</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">message</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">default</span>
<span class="c1">#-- statusbar --#</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">utf8</span><span class="w"> </span><span class="n">on</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">interval</span><span class="w"> </span><span class="mi">1</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">keys</span><span class="w"> </span><span class="n">vi</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">justify</span><span class="w"> </span><span class="n">left</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">display</span><span class="o">-</span><span class="n">time</span><span class="w"> </span><span class="mi">3000</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">default</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">white</span>
<span class="n">set</span><span class="o">-</span><span class="n">window</span><span class="o">-</span><span class="n">option</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">window</span><span class="o">-</span><span class="n">status</span><span class="o">-</span><span class="n">current</span><span class="o">-</span><span class="n">attr</span><span class="w"> </span><span class="n">default</span>
<span class="n">set</span><span class="o">-</span><span class="n">window</span><span class="o">-</span><span class="n">option</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">window</span><span class="o">-</span><span class="n">status</span><span class="o">-</span><span class="n">current</span><span class="o">-</span><span class="n">fg</span><span class="w"> </span><span class="n">red</span>
<span class="n">set</span><span class="o">-</span><span class="n">window</span><span class="o">-</span><span class="n">option</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">window</span><span class="o">-</span><span class="n">status</span><span class="o">-</span><span class="n">current</span><span class="o">-</span><span class="n">bg</span><span class="w"> </span><span class="n">default</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">left</span><span class="o">-</span><span class="n">length</span><span class="w"> </span><span class="mi">15</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">right</span><span class="o">-</span><span class="n">length</span><span class="w"> </span><span class="mi">55</span>
<span class="c1">#set -g status-left "#[fg=white,bg=blue] > #I #W < #[default] |" # 0:bash</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">left</span><span class="w"> </span><span class="s2">"#[fg=white,bg=default] > #S < #[default] |"</span><span class="w"> </span><span class="c1"># session-name</span>
<span class="n">set</span><span class="w"> </span><span class="o">-</span><span class="n">g</span><span class="w"> </span><span class="n">status</span><span class="o">-</span><span class="n">right</span><span class="w"> </span><span class="s2">"#[fg=red,bright][ #[fg=cyan]#H #[fg=red]]#[default] #[fg=yellow,bright]- %Y.%m.</span><span class="si">%d</span><span class="s2"> #[fg=green]%H:%M #[default]#[fg=magenta,bright](load: #(cat /proc/loadavg | cut -d </span><span class="se">\"</span><span class="s2"> </span><span class="se">\"</span><span class="s2"> -f 1,2,3))#[default]"</span>
</code></pre></div>
<h3 id="_10">使用</h3>
<p>我使用terminator配合tmux使用,更改terminator的配置(~/.config/terminator/conf)</p>
<div class="highlight"><pre><span></span><code><span class="k">[profiles]</span>
<span class="w"> </span><span class="k">[[default]]</span>
<span class="w"> </span><span class="na">login_shell</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">True</span>
<span class="w"> </span><span class="na">use_custom_command</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">True</span><span class="w"> </span><span class="c1"># 允许自定义命令</span>
<span class="w"> </span><span class="na">custom_command</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">tmux -2</span><span class="w"> </span><span class="c1"># 打开终端时执行tmux(-2 是强制终端使用256颜色) </span>
<span class="w"> </span><span class="na">background_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">transparent</span>
<span class="w"> </span><span class="na">scrollbar_position</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">hidden</span>
<span class="w"> </span><span class="na">foreground_color</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"</span><span class="c1">#ffffff"</span>
<span class="w"> </span><span class="na">show_titlebar</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">False</span>
<span class="w"> </span><span class="na">background_darkness</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">0.2</span>
</code></pre></div>
<h3 id="_11">常用快捷键:</h3>
<div class="highlight"><pre><span></span><code>Ctrl+b - 创建一个水平分隔
Ctrl+b | 创建一个垂直分隔
Ctrl+b j/k/h/l 上下左右切换分割窗
Ctrl+b Ctrl+j/k/h/l 向上下左右扩展分割窗大小
Ctrl+b c 创建一个窗口
Ctrl+b , 重命名窗口
Ctrl+b n 切换到下一个窗口
Ctrl+b p 切换到前一个窗口
</code></pre></div>
<h2 id="gnome-do">Gnome-Do</h2>
<p>Gnome Do 之前博客里有介绍请看<a href="/2012/9/1/Linux桌面高效工作----使用Gnome-DO/">这里</a>,由于Gonme Do使用Win+Space快捷键已经被Awesome占用,所以为了正常使用Gnome Do 我将Gnome Do的快捷键绑定到<code>Win+G</code>.</p>AppArmor引起的无法启动Evince2012-11-15T16:05:00+08:002012-11-15T16:05:00+08:00coldtag:www.linuxzen.com,2012-11-15:/apparmoryin-qi-de-wu-fa-qi-dong-evince.html<p>今天在Ubuntu上使用文档查看器(Evince),总是打开没有响应,在命令行下使用命令:
evince
却提示:</p>
<div class="highlight"><pre><span></span><code>No protocol specified
Cannot parse arguments: 无法打开显示:
</code></pre></div>
<p>google说是$XAUTHORITY权限的问题,于是查看:</p>
<div class="highlight"><pre><span></span><code>ls<span class="w"> </span>-l<span class="w"> </span><span class="nv">$XAUTHORITY</span>
-rw-------<span class="w"> </span><span class="m">1</span><span class="w"> </span>vim …</code></pre></div><p>今天在Ubuntu上使用文档查看器(Evince),总是打开没有响应,在命令行下使用命令:
evince
却提示:</p>
<div class="highlight"><pre><span></span><code>No protocol specified
Cannot parse arguments: 无法打开显示:
</code></pre></div>
<p>google说是$XAUTHORITY权限的问题,于是查看:</p>
<div class="highlight"><pre><span></span><code>ls<span class="w"> </span>-l<span class="w"> </span><span class="nv">$XAUTHORITY</span>
-rw-------<span class="w"> </span><span class="m">1</span><span class="w"> </span>vim<span class="w"> </span>vim<span class="w"> </span><span class="m">51</span><span class="w"> </span><span class="m">2012</span>-11-15<span class="w"> </span><span class="m">12</span>:12<span class="w"> </span>/data/home/vim/.Xauthority
</code></pre></div>
<p>更改为如下:</p>
<div class="highlight"><pre><span></span><code>chmod<span class="w"> </span>+rw<span class="w"> </span><span class="nv">$XAUTHORITY</span>
</code></pre></div>
<p>还是不行,这时候猛然想起查看日志,日志有如下一行:</p>
<div class="highlight"><pre><span></span><code>Nov 15 15:48:53 Vostro kernel: [13010.203241] type=1400 audit(1352965733.221:74): apparmor="DENIED" operation="open" parent=1 profile="/usr/bin/evince" name="/data/home/vim/.Xauthority" pid=7088 comm="evince" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000
</code></pre></div>
<p>好吧,问题是一个叫做AppArmor的内核模块引起的,它阻止了evince读取.Xauthority,找到配置文件,添加:</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>vim<span class="w"> </span>/etc/apparmor.d/usr.bin.evince<span class="w"> </span>
@<span class="o">{</span>HOME<span class="o">}</span>/.Xauthority<span class="w"> </span>rw,
sudo<span class="w"> </span>/etc/init.d/apparmor<span class="w"> </span>restart
</code></pre></div>
<p>做完这些还是刷同样的日志,无奈只有禁用它了:</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>ln<span class="w"> </span>-sf<span class="w"> </span>/etc/apparmor.d/usr.bin.evince<span class="w"> </span>/etc/apparmor.d/disable/
</code></pre></div>
<p>重启apparmor就能打开Evince了,都不知道这玩意是怎么加上的,之前都没问题.</p>
<p>此次解决看来Linux问题解决之路是从日志开始的,长时间Linux桌面工作差点都给忘掉日志的重要性.</p>python 使用pyxmpp2编写gtalk群2012-10-29T14:48:00+08:002012-10-29T14:48:00+08:00coldtag:www.linuxzen.com,2012-10-29:/python-shi-yong-pyxmpp2bian-xie-gtalkqun.html<p>gtalk是一款google开发的基于xmpp协议的聊天软件,其优点就是协议开源,我们可以通过任何支持xmpp的客户端协议来链接gtalk,但是gtalk不支持群聊天,所以各路高手都会自己来开发一个机器人来支持群功能.</p>
<p>其实 …</p><p>gtalk是一款google开发的基于xmpp协议的聊天软件,其优点就是协议开源,我们可以通过任何支持xmpp的客户端协议来链接gtalk,但是gtalk不支持群聊天,所以各路高手都会自己来开发一个机器人来支持群功能.</p>
<p>其实主要原理就是机器人接收到消息后再将消息广播出去,从而达到群的效果.</p>
<p>python有两个模块可以用来支持xmpp,分别是pyxmpp和pyxmpp2,之前也用pyxmpp写了一个,功能和兼容性不是很好,经常出现问题,所以又使用pyxmpp2重写了一遍,今天修复了一些bug,所以公布出来,大家可以下载测试,也可以加入我们使用gtalk进行群交流,</p>
<p>我们的gtalk机器人是:<code>clubot@vim-cn.com</code></p>
<p>喜欢gtalk,同时喜欢Linux/Python/Vim等爱好者的同学可以加进来交流</p>
<p>如果对我们的代码比较感兴趣,可以访问github,我们将代码放在了github上:https://github.com/coldnight/clubot</p>
<p>下面介绍一下安装:</p>
<p>环境为:python 2.7, 因为有少量的shell所以系统需要Linux,也可稍作更改支持windows</p>
<p>下载源码:</p>
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git://github.com/coldnight/clubot.git
</code></pre></div>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>easy_install<span class="w"> </span>pyxmpp2
</code></pre></div>
<p>修改settings.py文件,填入bot的账户和密码,执行:</p>
<div class="highlight"><pre><span></span><code>python<span class="w"> </span>clubot.py
</code></pre></div>
<p>即可开启群bot,群bot支持翻译,天气查询,贴代码等等待功能.</p>
<p>如果您想贡献代码可以加入我们的bot群:<code>clubot@vim-cn.com</code></p>
<p>如果您有bug可以提交在评论里</p>Linux高效工作----平铺式窗体管理器Awesome2012-10-25T10:23:00+08:002012-10-25T10:23:00+08:00coldtag:www.linuxzen.com,2012-10-25:/linuxgao-xiao-gong-zuo-ping-pu-shi-chuang-ti-guan-li-qi-awesome.html<p>在Linux桌面环境下开发,总想更高效的工作,我已经装备了Gnome Do和terminator,但是我还是觉得不够快我更加希望能解放右手(当然不是找个妹子戒撸,只是 …</p><p>在Linux桌面环境下开发,总想更高效的工作,我已经装备了Gnome Do和terminator,但是我还是觉得不够快我更加希望能解放右手(当然不是找个妹子戒撸,只是右手的鼠标),而且terminator在跑的东西过多的时候开多个terminator不太好管理,这时候一个词进入了我的眼睛平铺式窗体管理器,与传统窗体管理器不同的是平铺式窗体管理器的窗口不会重叠,窗口会被自动调整成正好铺满全屏的尺寸,也就是说无论开多少窗口都会把屏幕占满,如果想象力贫乏就装一个试试吧:</p>
<p>Awesome是一款运行在Unix和类Unix(Linux/FreeBSD)等系统上的一款平铺式窗体管理器,有占用资源小,易于管理和操作等等有点,这里不罗嗦这些说说安装,Ubuntu安装很简单</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>awesome
</code></pre></div>
<p>安装好后登出会话选择awesome登录,然后你是否茫然无知没办法工作了?先简单介绍下使用方法:</p>
<p>按<code>Win键+1~9</code>可以切换桌面,</p>
<p>没有菜单对吧 其实再右上角点一下就会出来一个菜单,打开程序会发现标题栏状态栏什么都木有了大大节省了桌面空间,可问题来了,怎么关闭啊不用担心</p>
<p>按 <code>Win键+Shift+C</code>可以关闭当前窗口</p>
<p>打开默认终端 <code>Win键+Enter</code>就可以打开终端</p>
<p>可以按住 <code>Win键+Shift+数字</code> 可以将当前窗口发送到相应的工作区</p>
<p>同样可以切换工作区的是 <code>Win+j/Win+k</code></p>
<p><code>Win+Space</code>可以切换当前工作区布局</p>
<p>是不是很高效!是不是解放了右手,问题来了,我的<code>gnome-do</code>无法用了,<code>win+space</code>被占了阿,好吧我把<code>Gnome-Do</code>的快捷键调成<code>Alt+Space</code>,</p>
<p>但是如何添加开机启动程序呢,很简单Awesome的配置文件是一个lua脚本(我不懂),可以在这个脚本里添加启动程序</p>
<p>创建配置文件:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>~/
mkdir<span class="w"> </span>.config/awesome/
cp<span class="w"> </span>/etc/xdg/awesome/rc.lua<span class="w"> </span>.config/awesome/
</code></pre></div>
<p>编辑配置文件在文件末尾添加:</p>
<div class="highlight"><pre><span></span><code><span class="n">autorun</span> <span class="o">=</span> <span class="kc">true</span>
<span class="n">autorunApps</span> <span class="o">=</span>
<span class="p">{</span>
<span class="s2">"pidgin"</span><span class="p">,</span>
<span class="s2">"thunderbird"</span><span class="p">,</span>
<span class="s2">"gnome-do"</span><span class="p">,</span>
<span class="s2">"guake"</span><span class="p">,</span>
<span class="s2">"/opt/qq2012/wineapp/qq/qq.sh"</span><span class="p">,</span>
<span class="p">}</span>
<span class="kr">if</span> <span class="n">autorun</span> <span class="kr">then</span>
<span class="kr">for</span> <span class="n">app</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="o">#</span><span class="n">autorunApps</span> <span class="kr">do</span>
<span class="n">awful</span><span class="p">.</span><span class="n">util</span><span class="p">.</span><span class="n">spawn_with_shell</span><span class="p">(</span><span class="n">autorunApps</span><span class="p">[</span><span class="n">app</span><span class="p">])</span>
<span class="kr">end</span>
<span class="kr">end</span>
</code></pre></div>
<p>很简单吧,应该看得出在哪里添加启动的程序吧,这里启动了<code>pidgin</code>,<code>雷鸟</code>,<code>gnome-do</code>,<code>guake</code>和<code>wine</code>的<code>qq2012</code></p>
<p>还有很多强大的功能这里就不在介绍配置方法,可以自行google之</p>Python将汉字按拼音排序--一个多音字引发的悲剧2012-10-24T18:29:00+08:002012-10-24T18:29:00+08:00coldtag:www.linuxzen.com,2012-10-24:/pythonjiang-yi-zi-an-pin-yin-pai-xu-yi-ge-duo-yin-zi-yin-fa-de-bei-ju.html<p>今天同事告诉我一个项目需要将汉字按拼音排序,之前没做过啊,就google之,down了一份汉字与拼音的对照表,对照表格式如下:</p>
<div class="highlight"><pre><span></span><code>拼 pin1
音 yin1
</code></pre></div>
<p>将文件保存为<code>pinyindata</code>,于是 …</p><p>今天同事告诉我一个项目需要将汉字按拼音排序,之前没做过啊,就google之,down了一份汉字与拼音的对照表,对照表格式如下:</p>
<div class="highlight"><pre><span></span><code>拼 pin1
音 yin1
</code></pre></div>
<p>将文件保存为<code>pinyindata</code>,于是有了如下对应的python代码:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold night</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 12-10-24 下午3:13</span>
<span class="c1">#</span>
<span class="kn">from</span> <span class="nn">os</span> <span class="kn">import</span> <span class="n">path</span>
<span class="kn">import</span> <span class="nn">locale</span>
<span class="k">class</span> <span class="nc">SortedByPy</span><span class="p">:</span>
<span class="n">__PYDICT__</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="n">PYDATA_PATH</span> <span class="o">=</span> <span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="vm">__file__</span><span class="p">),</span> <span class="s1">'pinyindata'</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">__PYDICT__</span><span class="p">:</span>
<span class="n">py_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">PYDATA_PATH</span><span class="p">,</span> <span class="s1">'r'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">py_file</span><span class="o">.</span><span class="n">readlines</span><span class="p">():</span>
<span class="n">word</span><span class="p">,</span> <span class="n">pinyin</span> <span class="o">=</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">__PYDICT__</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="n">word</span><span class="p">:</span><span class="n">pinyin</span><span class="o">.</span><span class="n">strip</span><span class="p">()})</span>
<span class="n">py_file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">get_py</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">word</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> 获取汉字拼音</span>
<span class="sd"> """</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">word</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__PYDICT__</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">i</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
<span class="k">return</span> <span class="n">result</span>
<span class="k">def</span> <span class="nf">cmp_by_py</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> 使用拼音比较两个汉字的先后(使用倒序)</span>
<span class="sd"> """</span>
<span class="n">r1</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_py</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
<span class="n">r2</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_py</span><span class="p">(</span><span class="n">B</span><span class="p">)</span>
<span class="k">return</span> <span class="n">cmp</span><span class="p">(</span><span class="n">r1</span><span class="p">,</span> <span class="n">r2</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sort</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">iterable</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> - `iterable` :</span>
<span class="sd"> - `key` : 有则对列表中的字典对应key的值进行排序</span>
<span class="sd"> """</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span><span class="n">v</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">iterable</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">v</span><span class="p">[</span><span class="n">key</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="k">if</span> <span class="n">key</span> <span class="k">else</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">if</span> <span class="n">f</span> <span class="o"><=</span> <span class="s1">'z'</span><span class="p">:</span><span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">iterable</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">i</span><span class="p">))</span>
<span class="k">if</span> <span class="n">key</span><span class="p">:</span>
<span class="n">result</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">cmp</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">cmp_by_py</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">key</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="n">key</span><span class="p">]),</span> <span class="n">reverse</span><span class="o">=</span> <span class="n">reverse</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">result</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="k">if</span> <span class="n">key</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">iterable</span><span class="p">,</span> <span class="n">cmp</span><span class="o">=</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">cmp_by_py</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">key</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="n">key</span><span class="p">]),</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">iterable</span><span class="p">,</span> <span class="n">cmp</span><span class="o">=</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">cmp_by_py</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span><span class="n">l</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
<span class="k">return</span> <span class="n">result</span>
</code></pre></div>
<p>嗯,写好后测试正常,提交后告诉同事,同事一测发现重庆怎么跑到最后啊,不对啊,有bug啊,我重新将重庆加入到我的测试,真是不行阿,好吧可能是对照表的问题,打开找了下,能找到重啊,难道是decode出问题了,于是打开咱的ipython shell,结果输入重庆,一回车终端崩了,好吧,我再试还是崩,于是我想可能是utf-8的bug或者python的uft-8的bug,于是我将我的想法告知了同事,同事也打开了<code>ipython试</code>了下,结果他的可以正常输入,正常decode/encode,这下我想可能是方法的问题吧.</p>
<p>于是看看有无别的办法,看到一个对Linux有效可能对window有效的办法,好嘛正好咱也Linux果断拿过来试试,于是下面的代码诞生了:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1"># Author : cold night</span>
<span class="c1"># E-mail : wh_linux@126.com</span>
<span class="c1"># Date : 12-10-24 下午3:13</span>
<span class="c1">#</span>
<span class="c1">#from os import path</span>
<span class="kn">import</span> <span class="nn">locale</span>
<span class="k">class</span> <span class="nc">SortedByPy</span><span class="p">:</span>
<span class="n">__PYDICT__</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="c1">#PYDATA_PATH = path.join(path.dirname(__file__), 'pinyindata')</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> if not self.__PYDICT__:</span>
<span class="sd"> py_file = open(self.PYDATA_PATH, 'r')</span>
<span class="sd"> for line in py_file.readlines():</span>
<span class="sd"> word, pinyin = line.split('\t')</span>
<span class="sd"> self.__PYDICT__.update({word:pinyin.strip()})</span>
<span class="sd"> py_file.close()</span>
<span class="sd"> """</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">get_py</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">word</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> 获取汉字拼音</span>
<span class="sd"> """</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">word</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__PYDICT__</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">i</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
<span class="k">return</span> <span class="n">result</span>
<span class="k">def</span> <span class="nf">cmp_by_py</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> 使用拼音比较两个汉字的先后(使用倒序)</span>
<span class="sd"> """</span>
<span class="n">r1</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_py</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
<span class="n">r2</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_py</span><span class="p">(</span><span class="n">B</span><span class="p">)</span>
<span class="k">return</span> <span class="n">cmp</span><span class="p">(</span><span class="n">r1</span><span class="p">,</span> <span class="n">r2</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sort</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">iterable</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> - `iterable` :</span>
<span class="sd"> - `key` : 有则对列表中的字典对应key的值进行排序</span>
<span class="sd"> """</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> for i,v in enumerate(iterable):</span>
<span class="sd"> f = v[key][0] if key else v[0]</span>
<span class="sd"> if f <= 'z':result.append(iterable.pop(i))</span>
<span class="sd"> if key:</span>
<span class="sd"> result.sort(cmp = lambda x, y: self.cmp_by_py(x[key], y[key]), reverse= reverse)</span>
<span class="sd"> else:</span>
<span class="sd"> result.sort(reverse=reverse)</span>
<span class="sd"> """</span>
<span class="n">locale</span><span class="o">.</span><span class="n">setlocale</span><span class="p">(</span><span class="n">locale</span><span class="o">.</span><span class="n">LC_ALL</span><span class="p">,</span> <span class="s1">'zh_CN.UTF-8'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">key</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">iterable</span><span class="p">,</span> <span class="n">cmp</span><span class="o">=</span><span class="n">locale</span><span class="o">.</span><span class="n">strcoll</span><span class="p">,</span> <span class="n">key</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="n">key</span><span class="p">],</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">iterable</span><span class="p">,</span> <span class="n">cmp</span><span class="o">=</span><span class="n">locale</span><span class="o">.</span><span class="n">strcoll</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
<span class="k">return</span> <span class="n">result</span>
</code></pre></div>
<p>写好了跑起来测试嘛,重庆还是躺在了最后,我想这应该就是utf-8的bug了吧,系统都这样了,于是和同事交流后认为也许是bug吧,回到座位后我苦思冥想阿,绞尽脑汁啊,看看有没有别的办法阿,突然一道闪电阿,我明白了,重<code>多音字</code>啊也读<code>zhong</code>啊,排在最后是对的啊,没有错误,也没有bug,一个<code>多音字</code>引发的悲剧哇.</p>部署Tornado时iptables引发的的一个问题2012-09-29T19:11:00+08:002012-09-29T19:11:00+08:00coldtag:www.linuxzen.com,2012-09-29:/bu-shu-tornadoshi-iptablesyin-fa-de-de-yi-ge-wen-ti.html<p>今天在CentOS上部署了一个Tornado,使用nginx做代理,</p>
<p>tornado使用8888,端口,使用nginx作为反向代理,配置文件如下:</p>
<div class="highlight"><pre><span></span><code><span class="k">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">listen</span><span class="w"> </span><span class="mi">80</span><span class="p">;</span>
<span class="w"> </span><span class="kn">server_name</span><span class="w"> </span><span class="s">www.linuxzen.com</span><span class="p">;</span>
<span class="w"> </span><span class="kn">location</span><span class="w"> </span><span class="s">/</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">proxy_pass_header</span><span class="w"> </span><span class="s">Server</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_redirect</span><span class="w"> </span><span class="no">off</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_set_header</span><span class="w"> </span><span class="s">X-Real_IP</span><span class="w"> </span><span class="nv">$remote_addr</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_set_header</span><span class="w"> </span><span class="s">X-Scheme</span><span class="w"> </span><span class="nv">$scheme</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_pass</span><span class="w"> </span><span class="s">http://127.0.0.1:8888</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>iptables filter表的INPUT链是DROP的,所以添加 …</p><p>今天在CentOS上部署了一个Tornado,使用nginx做代理,</p>
<p>tornado使用8888,端口,使用nginx作为反向代理,配置文件如下:</p>
<div class="highlight"><pre><span></span><code><span class="k">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">listen</span><span class="w"> </span><span class="mi">80</span><span class="p">;</span>
<span class="w"> </span><span class="kn">server_name</span><span class="w"> </span><span class="s">www.linuxzen.com</span><span class="p">;</span>
<span class="w"> </span><span class="kn">location</span><span class="w"> </span><span class="s">/</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kn">proxy_pass_header</span><span class="w"> </span><span class="s">Server</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_redirect</span><span class="w"> </span><span class="no">off</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_set_header</span><span class="w"> </span><span class="s">X-Real_IP</span><span class="w"> </span><span class="nv">$remote_addr</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_set_header</span><span class="w"> </span><span class="s">X-Scheme</span><span class="w"> </span><span class="nv">$scheme</span><span class="p">;</span>
<span class="w"> </span><span class="kn">proxy_pass</span><span class="w"> </span><span class="s">http://127.0.0.1:8888</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>iptables filter表的INPUT链是DROP的,所以添加如下规则:</p>
<div class="highlight"><pre><span></span><code>iptables<span class="w"> </span>-A<span class="w"> </span>INPUT<span class="w"> </span>-p<span class="w"> </span>tcp<span class="w"> </span>-s<span class="w"> </span><span class="m">127</span>.0.0.1<span class="w"> </span>--dport<span class="w"> </span><span class="m">8888</span><span class="w"> </span>-j<span class="w"> </span>ACCEPT
</code></pre></div>
<p>但是访问nginx总是返回502 Bad Gateway.一开始不认为是防火墙的问题,所以百思不得其解,各种问题试过都不行之后,后来想想tornado返回数据同样也是一条进站,所以还需要添加一条</p>
<div class="highlight"><pre><span></span><code>iptables<span class="w"> </span>-A<span class="w"> </span>INPUT<span class="w"> </span>-p<span class="w"> </span>tcp<span class="w"> </span>-s<span class="w"> </span><span class="m">127</span>.0.0.1<span class="w"> </span>--dport<span class="w"> </span><span class="m">8888</span><span class="w"> </span>-j<span class="w"> </span>ACCEPT
</code></pre></div>
<p>至此,通过nginx才能正常访问tornado</p>用Vim为Python源码自动添加#!行和编码行2012-09-29T18:56:00+08:002012-09-29T18:56:00+08:00coldtag:www.linuxzen.com,2012-09-29:/yong-vimwei-pythonyuan-ma-zi-dong-tian-jia-xing-he-bian-ma-xing.html<p>每次开始写Python打开文件第一件事就是写上<code>#!/usr/bin/env python和</code>编码之类的东西,</p>
<p>太多了,写烦就,写了一个打开Python自动填充的函数,将下面内容添加到<code>~/.vimrc</code>下即可每次打开如果没有 …</p><p>每次开始写Python打开文件第一件事就是写上<code>#!/usr/bin/env python和</code>编码之类的东西,</p>
<p>太多了,写烦就,写了一个打开Python自动填充的函数,将下面内容添加到<code>~/.vimrc</code>下即可每次打开如果没有上述行则会自动填充:</p>
<div class="highlight"><pre><span></span><code><span class="k">function</span> InsertPythonHeader<span class="p">()</span>
<span class="k">let</span> l1 <span class="p">=</span> getline<span class="p">(</span><span class="m">1</span><span class="p">)</span>
<span class="k">let</span> l2 <span class="p">=</span> getline<span class="p">(</span><span class="m">2</span><span class="p">)</span>
<span class="k">if</span> <span class="k">match</span><span class="p">(</span><span class="s1">'\#!/'</span><span class="p">,</span> l1<span class="p">)</span> <span class="p">==</span> <span class="m">0</span>
exec <span class="m">1</span>
normal O
<span class="k">call</span> setline<span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="s1">'#!/usr/bin/env python'</span><span class="p">)</span>
<span class="k">endif</span>
<span class="k">if</span> <span class="k">match</span><span class="p">(</span><span class="s2">"\#"</span><span class="p">,</span> l2<span class="p">)</span> <span class="p">==</span> <span class="m">0</span> && <span class="p">(</span><span class="k">match</span><span class="p">(</span><span class="s2">"-"</span><span class="p">,</span> l2<span class="p">)</span> <span class="p">!=</span> <span class="m">2</span> ¦¦ <span class="p">(</span><span class="k">match</span><span class="p">(</span><span class="s2">"code"</span><span class="p">,</span> l2<span class="p">)</span> <span class="p">!=</span> <span class="m">2</span><span class="p">))</span>
exec <span class="m">2</span>
normal O
<span class="k">call</span> setline<span class="p">(</span><span class="m">2</span><span class="p">,</span><span class="s1">'#-*- coding:utf-8 -*-'</span><span class="p">)</span>
<span class="k">endif</span>
<span class="k">endfunction</span>
<span class="k">au</span> <span class="nb">FileType</span> python <span class="k">call</span> InsertPythonHeader<span class="p">()</span>
</code></pre></div>分享Vim两种好用的功能:状态行和空白字符可见2012-09-29T17:30:00+08:002012-09-29T17:30:00+08:00coldtag:www.linuxzen.com,2012-09-29:/fen-xiang-vimliang-chong-hao-yong-de-gong-neng-zhuang-tai-xing-he-kong-bai-zi-fu-ke-jian.html<p>马上要放假了,没事折腾了一下Vim发现了两个非常棒的功能,一个是给Vim添加一个状态栏,一个是可以在编辑的时候显示空白,</p>
<p>我的状态栏显示了:
正在编辑的文件名 …</p><p>马上要放假了,没事折腾了一下Vim发现了两个非常棒的功能,一个是给Vim添加一个状态栏,一个是可以在编辑的时候显示空白,</p>
<p>我的状态栏显示了:
正在编辑的文件名,
选项
是Git显示git分支(需要fugitive插件)
文件类型
当前目录
当前字符的ASCII和16进制码
右边是当前光标所在行/列,文件的位置的百分比,和文件的长度
fugitive 可以在git获得:</p>
<div class="highlight"><pre><span></span><code>git clone http://github.com/tpope/vim-fugitive.git
</code></pre></div>
<p>将plugin目录下的fugitive.vim复制到~/.vim/plugin下
在~/.vimrc添加如下内容</p>
<div class="highlight"><pre><span></span><code><span class="k">if</span> has<span class="p">(</span><span class="s1">'statusline'</span><span class="p">)</span>
<span class="k">set</span> <span class="nb">laststatus</span><span class="p">=</span><span class="m">2</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">=</span>%<span class="p"><</span>%<span class="k">f</span>\ <span class="c">" 文件名</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>%<span class="k">w</span>%<span class="k">h</span>%<span class="k">m</span>%<span class="k">r</span> <span class="c">" 选项</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>%{fugitive#<span class="nb">statusline</span><span class="p">()</span>} <span class="c">"Git</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>\ [%{&<span class="nb">ff</span>}/%Y] <span class="c">" filetype</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>\ [%{getcwd<span class="p">()</span>}] <span class="c">" current dir</span>
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>\ [A<span class="p">=</span>\%<span class="m">03</span>.<span class="m">3</span>b<span class="sr">/H=\%02.2B] " ASCII /</span> Hexadecimal value of char
<span class="k">set</span> <span class="nb">statusline</span><span class="p">+=</span>%<span class="p">=</span>%<span class="m">-14</span>.<span class="p">(</span>%<span class="k">l</span><span class="p">,</span>%<span class="k">c</span>%V%<span class="p">)</span>\ %<span class="k">p</span>%%\ %L <span class="c">" Right aligned file nav info</span>
<span class="k">endif</span>
</code></pre></div>
<p>要想在编辑时将空白可见可以在.vimrc中添加如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="k">set</span> <span class="nb">listchars</span><span class="p">=</span><span class="k">tab</span>:<span class="p">>-,</span>trail:<span class="p">-,</span>extends:#<span class="p">,</span>nbsp:<span class="p">-</span>
</code></pre></div>
<p>即可tab显示为>---,空格显示-,行尾的空白显示-,</p>
<p>设置完后非常的cool,上图一张:
<img alt="vim空白符" src="/upload/Screenshot-2012-09-29-173517.png"></p>用Python将绝对URL替换成相对URL2012-09-29T15:52:00+08:002012-09-29T15:52:00+08:00coldtag:www.linuxzen.com,2012-09-29:/yong-pythonjiang-jue-dui-urlti-huan-cheng-xiang-dui-url.html<p>公司一个项目需要上传图片,一开始同事将图片上传后结合当前主机拼成了一个绝对的URL(<a href="#">http://192.168.1.1:888/m/getimg?filename=xxx.jpg</a>由 …</p><p>公司一个项目需要上传图片,一开始同事将图片上传后结合当前主机拼成了一个绝对的URL(<a href="#">http://192.168.1.1:888/m/getimg?filename=xxx.jpg</a>由于同时给手机终端提供接口,在手机终端会引起一些bug,改完代码后要求将以前的uri替换成相对的URL(/m/getimg?filename=xxx.jpg),由于图片是用img标签嵌入到内容同时用a标签括起显示大图的,所以需要读取数据库并对内容进行替换,</p>
<p>脚本内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1">#-*- coding:utf-8 -*-</span>
<span class="c1">#</span>
<span class="c1">#</span>
<span class="c1"># author : cold night</span>
<span class="c1"># email : wh_linux@126.com</span>
<span class="c1">#</span>
<span class="kn">import</span> <span class="nn">pymongo</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">from</span> <span class="nn">StringIO</span> <span class="kn">import</span> <span class="n">StringIO</span>
<span class="n">conn</span> <span class="o">=</span> <span class="n">pymongo</span><span class="o">.</span><span class="n">Connection</span><span class="p">()</span>
<span class="n">db</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">test</span>
<span class="k">def</span> <span class="nf">replace_url</span><span class="p">():</span>
<span class="n">regex</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s1">'([href¦src])=["¦</span><span class="se">\'</span><span class="s1">]http://.*?(/m/getimg\?.*?)["¦</span><span class="se">\'</span><span class="s1">]'</span><span class="p">)</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">db</span><span class="p">[</span><span class="s1">'test'</span><span class="p">]</span><span class="o">.</span><span class="n">find</span><span class="p">()</span>
<span class="n">db_coll</span> <span class="o">=</span> <span class="n">db</span><span class="p">[</span><span class="s1">'test'</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">replace</span><span class="p">(</span><span class="n">r</span><span class="p">):</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'content'</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">content</span><span class="p">:</span> <span class="k">return</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">StringIO</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="n">content</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">StringIO</span><span class="p">()</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">content</span><span class="o">.</span><span class="n">readlines</span><span class="p">():</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">regex</span><span class="o">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s1">'\1="\2"'</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
<span class="n">result</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">if</span> <span class="n">content</span><span class="p">:</span>
<span class="n">r</span><span class="p">[</span><span class="s1">'content'</span><span class="p">]</span> <span class="o">=</span> <span class="n">content</span>
<span class="n">_id</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'_id'</span><span class="p">)</span>
<span class="n">db_coll</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s1">'_id'</span><span class="p">:</span><span class="n">_id</span><span class="p">},</span> <span class="n">r</span><span class="p">)</span>
<span class="n">results</span> <span class="o">=</span> <span class="p">[</span><span class="n">replace</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">results</span><span class="p">]</span>
<span class="k">if</span> <span class="vm">__name__</span><span class="o">==</span><span class="s2">"__main__"</span><span class="p">:</span><span class="n">replace_url</span><span class="p">()</span>
</code></pre></div>用Python对各种编程语言进行代码高亮2012-09-29T15:11:00+08:002012-09-29T15:11:00+08:00coldtag:www.linuxzen.com,2012-09-29:/yong-pythondui-ge-chong-bian-cheng-yu-yan-jin-xing-dai-ma-gao-liang.html<p>做了一个在线代码高亮的项目,因为写的Gtalk群Bot需要这个功能支持,贴到第三方怕被人给封,所以干脆想自己写一个,强大的Python一如既往没让我失望,一个强大的Pygments模块可以对多种(很多 …</p><p>做了一个在线代码高亮的项目,因为写的Gtalk群Bot需要这个功能支持,贴到第三方怕被人给封,所以干脆想自己写一个,强大的Python一如既往没让我失望,一个强大的Pygments模块可以对多种(很多)语言进行代码高亮</p>
<p>下面来介绍一下它:</p>
<p>首先安装很简单,使用easy_install来进行安装:</p>
<div class="highlight"><pre><span></span><code>easy_install<span class="w"> </span>pygments
</code></pre></div>
<p>安装完后我们来使用,Python的简单不会让大家失望:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">pygments.lexers</span> <span class="kn">import</span> <span class="n">PythonLexver</span>
<span class="kn">from</span> <span class="nn">pygments.formatters</span> <span class="kn">import</span> <span class="n">HtmlFormatter</span>
<span class="kn">from</span> <span class="nn">pygments</span> <span class="kn">import</span> <span class="n">highlight</span>
<span class="n">formatter</span> <span class="o">=</span> <span class="n">HtmlFormatter</span><span class="p">(</span><span class="n">encoding</span><span class="o">=</span><span class="s1">'utf-8'</span><span class="p">,</span> <span class="n">style</span> <span class="o">=</span> <span class="s1">'emacs'</span><span class="p">,</span> <span class="n">linenos</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span>
<span class="n">code</span> <span class="o">=</span> <span class="n">highlight</span><span class="p">(</span><span class="s1">'print "hello, world"'</span><span class="p">,</span> <span class="n">PythonLexer</span><span class="p">(),</span> <span class="n">formatter</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">code</span>
</code></pre></div>
<h5 id="_1">结果</h5>
<div class="highlight"><pre><span></span><code>'<span class="nt"><table</span><span class="w"> </span><span class="na">class=</span><span class="s">"highlighttable"</span><span class="nt">><tr><td</span><span class="w"> </span><span class="na">class=</span><span class="s">"linenos"</span><span class="nt">><div</span><span class="w"> </span><span class="na">class=</span><span class="s">"linenodiv"</span><span class="nt">><pre></span>1<span class="nt"></pre></div></td><td</span><span class="w"> </span><span class="na">class=</span><span class="s">"code"</span><span class="nt">><div</span><span class="w"> </span><span class="na">class=</span><span class="s">"highlight"</span><span class="nt">><pre><span</span><span class="w"> </span><span class="na">class=</span><span class="s">"k"</span><span class="nt">></span>print<span class="nt"></span></span><span class="w"> </span><span class="nt"><span</span><span class="w"> </span><span class="na">class=</span><span class="s">"s"</span><span class="nt">></span><span class="ni">&quot;</span>hello,<span class="w"> </span>world<span class="ni">&quot;</span><span class="nt"></span></span>\n<span class="nt"></pre></div></span>\n<span class="nt"></td></tr></table></span>'
</code></pre></div>
<p>这样就简单的对代码进行了高亮,当然如果你做了上面操作,然后把内容输入到一个文件里查看,肯定大呼坑爹,因为根本没高亮,因为默认是不会输出css的 我们还要获取css加入到html中去:</p>
<div class="highlight"><pre><span></span><code><span class="n">css</span> <span class="o">=</span> <span class="n">formatter</span><span class="o">.</span><span class="n">get_style_defs</span><span class="p">()</span>
</code></pre></div>
<p>然后把css内容和上面的html一起写入到html文件就可以看到高亮的代码了(千万不要告诉我你不知道css应该放在什么位置)</p>
<p>欢迎大家加入到我们的gtalk群来讨论Python/vim/Linux 或者蛋疼的时候聊聊人生,有很多好玩的功能等着大家.
使用gtalk添加:clubot@vim-cn.com</p>Python 优雅的操作字典2012-09-17T11:42:00+08:002012-09-17T11:42:00+08:00coldtag:www.linuxzen.com,2012-09-17:/python-you-ya-de-cao-zuo-zi-dian.html<p>Python 中的字典是Python中一个键值映射的数据结构,下面介绍一下如何优雅的操作字典.</p>
<h2 id="11">1.1 创建字典</h2>
<p>Python有两种方法可以创建字典,第一种是使用花括号,另一种是使用内建
函数dict</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info …</span></code></pre></div><p>Python 中的字典是Python中一个键值映射的数据结构,下面介绍一下如何优雅的操作字典.</p>
<h2 id="11">1.1 创建字典</h2>
<p>Python有两种方法可以创建字典,第一种是使用花括号,另一种是使用内建
函数dict</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{}</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
</code></pre></div>
<h2 id="12">1.2 初始化字典</h2>
<p>Python可以在创建字典的时候初始化字典</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"name"</span> <span class="p">:</span> <span class="s1">'cold'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span> <span class="o">=</span> <span class="s1">'cold'</span><span class="p">)</span> <span class="c1"># 更优雅</span>
</code></pre></div>
<p>很明显第二种方法更加的优雅和减少一些特殊字符的输入,但是有种情况第二种不能胜任</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">key</span> <span class="o">=</span> <span class="s1">'name'</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{</span> <span class="n">key</span> <span class="p">:</span><span class="s1">'cold'</span><span class="p">}</span> <span class="c1"># {'name':'cold'}</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">key</span> <span class="o">=</span> <span class="s1">'cold'</span><span class="p">)</span> <span class="c1"># {'key': 'cold'}</span>
</code></pre></div>
<p>明显第二种方法就会引发一个不容易找到的bug</p>
<p>Python字典还有一种初始化方式,就是使用字典的fromkeys方法可以从列表中获取元素作为键并用None或fromkeys方法的第二个参数初始化</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{}</span><span class="o">.</span><span class="n">fromkeys</span><span class="p">([</span><span class="s1">'name'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">])</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="kc">None</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span><span class="o">.</span><span class="n">fromkeys</span><span class="p">([</span><span class="s1">'name'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">])</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="kc">None</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span><span class="o">.</span><span class="n">fromkeys</span><span class="p">([</span><span class="s1">'name'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">],</span> <span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">}</span>
</code></pre></div>
<h2 id="13">1.3 优雅的获取键值</h2>
<p>字典可以这样获取到键的值</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'name'</span><span class="p">:</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">:</span><span class="s1">'linuxzen.com'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span>
<span class="s1">'cold'</span>
</code></pre></div>
<p>但是如果获取不存在的键的值就会触发的一个KeyError异常,字典有一个get方法,可以使用字典get方法更加优雅的获取字典</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span> <span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'www.linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'name'</span><span class="p">)</span>
<span class="s1">'cold'</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'blogname'</span><span class="p">)</span>
<span class="kc">None</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'blogname'</span><span class="p">,</span> <span class="s1">'linuxzen'</span><span class="p">)</span>
<span class="s1">'linuxzen'</span>
</code></pre></div>
<p>我们看到使用get方法获取不存在的键值的时候不会触发异常,同时get方法接收两个参数,当不存在该键的时候就会返回第二个参数的值
我们可以看到使用get更加的优雅</p>
<h2 id="14">1.4 更新/添加</h2>
<p>Python 字典可以使用键作为索引来访问/更新/添加值</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">info</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'cold'</span>
<span class="o">>>></span> <span class="n">info</span><span class="p">[</span><span class="s1">'blog'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'linuxzen.com'</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold night'</span><span class="p">}</span>
</code></pre></div>
<p>同时Python字典的update方法也可以更新和添加字典</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s1">'name'</span><span class="p">:</span><span class="s1">'cold night'</span><span class="p">,</span> <span class="s1">'blogname'</span><span class="p">:</span><span class="s1">'linuxzen'</span><span class="p">})</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold night'</span><span class="p">,</span> <span class="s1">'blogname'</span><span class="p">:</span> <span class="s1">'linuxzen'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'www.linuxzen.com'</span><span class="p">)</span> <span class="c1"># 更优雅</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'blogname'</span><span class="p">:</span> <span class="s1">'linuxzen'</span><span class="p">}</span>
</code></pre></div>
<p>Python字典的update方法可以使用一个字典来更新字典,也可以使用参数传递类似dict函数一样的方式更新一个字典,上面代码中哦功能的第二个更加优雅,但是同样和dict函数类似,键是变量时也只取字面值</p>
<h2 id="15">1.5 字典删除</h2>
<p>可以调用Python内置关键字del来删除一个键值</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold'</span><span class="p">}</span>
<span class="o">>>></span> <span class="k">del</span> <span class="n">info</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">}</span>
</code></pre></div>
<p>同时也可以使用字典的pop方法来取出一个键值,并删除</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s1">'name'</span><span class="p">)</span>
<span class="s1">'cold'</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'linuxzen.com'</span><span class="p">}</span>
</code></pre></div>
<h2 id="16">1.6 其他操作</h2>
<p>获取所有key</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
<span class="p">[</span><span class="s1">'blog'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">]</span>
</code></pre></div>
<p>获取key,value并循环</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'linuxzen.com'</span><span class="p">)</span>
<span class="o">>>></span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">info</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="o">...</span> <span class="nb">print</span> <span class="n">key</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="n">value</span>
<span class="o">...</span>
<span class="n">blog</span> <span class="p">:</span> <span class="n">linuxzen</span><span class="o">.</span><span class="n">com</span>
<span class="n">name</span> <span class="p">:</span> <span class="n">cold</span>
</code></pre></div>Python 字典和列表陷阱2012-09-17T10:22:00+08:002012-09-17T10:22:00+08:00coldtag:www.linuxzen.com,2012-09-17:/python-zi-dian-he-lie-biao-xian-jing.html<p>Python 中有三个非常好用的数据结构,列表,元组和字典,
元组是不可变的,列表可以保存任意类型的Python对象,并可以随意扩展没有大 …</p><p>Python 中有三个非常好用的数据结构,列表,元组和字典,
元组是不可变的,列表可以保存任意类型的Python对象,并可以随意扩展没有大小限制,
字典是一个key-value的键值映射的类型,可以存放任何Python对象,可以嵌套字典,
值可以是字典元组或者字典</p>
<p>这里说是Python 字典和列表的陷阱不如说是Python的一些特性,如果不了解这些特性
就会引发一些难以寻找的bug</p>
<p>下面我们来介绍这些特性</p>
<p>Python中所有对列表和字典的使用仅仅是对原来对象的引用而不是创建一个新的对象
如下面代码:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'cold'</span><span class="p">,</span> <span class="n">blog</span><span class="o">=</span><span class="s1">'www.linuxzen.com'</span><span class="p">)</span> <span class="c1"># 创建字典{'name':'cold', 'blog':'www.linuxzen.com'}</span>
<span class="o">>>></span> <span class="n">info2</span> <span class="o">=</span> <span class="n">info</span> <span class="c1"># 赋值给info2</span>
<span class="o">>>></span> <span class="n">info2</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'cold night'</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="o">>>></span> <span class="n">info2</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold night'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold night'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">names2</span> <span class="o">=</span> <span class="n">names</span>
<span class="o">>>></span> <span class="n">names2</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'cold night'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">names</span>
<span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen'</span><span class="p">,</span> <span class="s1">'cold night'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">names2</span>
<span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen'</span><span class="p">,</span> <span class="s1">'cold night'</span><span class="p">]</span>
</code></pre></div>
<p>大家看到如果将列表或者字典重新赋值给另外一个变量并没有达到预想的效果,
我们更改一个的同时另外一个也在同时更改,如果我们想保留一个快照,很明显我们
没有达到我们想要的效果,另外还有一种常见的使用,因为我们知道普通变量传递给
函数,函数在内部更改是不会影响到外部变量的,那么列表和字典呢?
我们来看如下代码,我们创建一个函数,是字典就添加一个键和值,是列表就在尾部添加一个元素</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="k">def</span> <span class="nf">add_something</span><span class="p">(</span><span class="n">info</span><span class="p">):</span>
<span class="o">...</span> <span class="k">if</span> <span class="nb">type</span><span class="p">(</span><span class="n">info</span><span class="p">)</span> <span class="o">==</span> <span class="nb">dict</span><span class="p">:</span>
<span class="o">...</span> <span class="n">info</span><span class="p">[</span><span class="s1">'msg'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'Hello,'</span><span class="o">+</span> <span class="n">info</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span>
<span class="o">...</span> <span class="k">elif</span> <span class="nb">type</span><span class="p">(</span><span class="n">info</span><span class="p">)</span> <span class="o">==</span> <span class="nb">list</span><span class="p">:</span>
<span class="o">...</span> <span class="n">info</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'add to the list'</span><span class="p">)</span>
<span class="o">...</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'name'</span><span class="p">:</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">:</span><span class="s1">'www.linuxzen.com'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">add_something</span><span class="p">(</span><span class="n">info</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'msg'</span><span class="p">:</span> <span class="s1">'Hello,cold'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen.com'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">add_something</span><span class="p">(</span><span class="n">names</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">names</span>
<span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'add to the list'</span><span class="p">]</span>
</code></pre></div>
<p>如上代码明显不是我们想要的结果,如果这个列表/字典仅仅用在一个地方可能不会发生什么
如果我们其他地方需要同样的列表进行处理,如果你不知道这个特性就会产生很难寻找的bug
当上面并不是我们想要的我们该如何避免上面呢,我们可以对列表/字典做一个拷贝,而不是
简单的引用</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen.com'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">names2</span> <span class="o">=</span> <span class="n">names</span><span class="p">[:]</span>
<span class="o">>>></span> <span class="n">names2</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'cold night'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">names</span>
<span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen.com'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">names2</span>
<span class="p">[</span><span class="s1">'cold'</span><span class="p">,</span> <span class="s1">'night'</span><span class="p">,</span> <span class="s1">'linuxzen.com'</span><span class="p">,</span> <span class="s1">'cold night'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">info</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'name'</span><span class="p">:</span><span class="s1">'cold night'</span><span class="p">,</span> <span class="s1">'blog'</span><span class="p">:</span><span class="s1">'www.linuxzen.com'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info2</span> <span class="o">=</span> <span class="n">info</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">info2</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'cold'</span>
<span class="o">>>></span> <span class="n">info</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold night'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">info2</span>
<span class="p">{</span><span class="s1">'blog'</span><span class="p">:</span> <span class="s1">'www.linuxzen.com'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">:</span> <span class="s1">'cold'</span><span class="p">}</span>
</code></pre></div>
<p>上面代码列表使用[:]可以创建一个列表的副本而不是引用
字典的copy方法同样可以创建一个字典的副本而不是引用
这样就可以避免之前所说的引用的情况</p>Linux桌面高效工作----使用Gnome DO2012-09-01T08:03:00+08:002012-09-01T08:03:00+08:00coldtag:www.linuxzen.com,2012-09-01:/linuxzhuo-mian-gao-xiao-gong-zuo-shi-yong-gnome-do.html<p>不知大家是否和我一样在win下系统win+r输入命令来快速启动程序,这两天在Linux下碰到一个比这更爽,更快的软件,<code>Gnome Do</code>.</p>
<p>Gnome Do能根据用户键入的内容进行自动匹配,从而快速打开系统中已有的程序、文件、书签等。不仅如此 …</p><p>不知大家是否和我一样在win下系统win+r输入命令来快速启动程序,这两天在Linux下碰到一个比这更爽,更快的软件,<code>Gnome Do</code>.</p>
<p>Gnome Do能根据用户键入的内容进行自动匹配,从而快速打开系统中已有的程序、文件、书签等。不仅如此,GNOME Do 还包括插件,从而能够做更多事,</p>
<p>比如你安装了pidgin插件只需输入联系人的名字即可打开与他/她的会话,安装了file这个插件输入文件/目录的名字即可打开目录或文件,</p>
<p>当然还有一个不足就是不支持中文</p>
<p>ubuntu用户可以按照下面安装:</p>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>gnome-do
</code></pre></div>
<p>启动之后Gnome do不会停留任务栏或通知栏只需按<code>Win(ubuntu下称为super)+Space</code>即可启动,输入你想启动的应用程序名字即可打开/关闭等操作.是不是很酷提高不少的工作效率</p>推荐两款不错的终端软件2012-08-29T09:36:00+08:002012-08-29T09:36:00+08:00coldtag:www.linuxzen.com,2012-08-29:/tui-jian-liang-kuan-bu-cuo-de-zhong-duan-ruan-jian.html<p>一直在Linux下做开发,一个好用的终端软件能帮你节省很多时间和精力</p>
<p>作为一个经常喜欢敲命令的人,可能要同时做很多操作,Linux各个桌面的窗口切换有多那啥,这 …</p><p>一直在Linux下做开发,一个好用的终端软件能帮你节省很多时间和精力</p>
<p>作为一个经常喜欢敲命令的人,可能要同时做很多操作,Linux各个桌面的窗口切换有多那啥,这里就不吐槽了,
我总是在想要做另外一个操作,但又不想结束当前的工作,之前我习惯于再打开一个终端,但是后来终端越来越多导致我想找回原来的工作的时候就变的很费力,而且对桌面有洁癖的人不允许任务栏太杂
后来发现了一款终端软件terminator,它支持分割终端,并可以在终端中快速切换.还有一款下拉式的终端软件Guake可以随意呼出隐藏.下面就一一介绍一下.</p>
<h2 id="1-terminator">1 安装Terminator</h2>
<div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>terminator
</code></pre></div>
<h2 id="2">2 使用</h2>
<p>打开Termintor按<code>Ctrl-E</code>(注意是大E要按住Shift)可以垂直分割终端
<code>Ctrl-O</code> 可水平分割终端
按住Alt然后按上下左右可以在不同的分割窗中切换
<code>Ctrl-D</code> 可以关闭分割窗</p>
<h3 id="21">2.1 配置</h3>
<p>terminator配置文件在<code>~/.config/terminator/config</code>
可以通过这个配置文件配置<code>terminator</code>的字体和颜色</p>
<div class="highlight"><pre><span></span><code><span class="n">font</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Monaco</span><span class="w"> </span><span class="mh">10</span><span class="w"> </span><span class="p">#</span><span class="err">设置体字</span>
<span class="n">background_color</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"#204070"</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">背景颜色</span>
<span class="n">foreground_color</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"#F0F0F0"</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">字体颜色</span>
<span class="n">cursor_blink</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">True</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">设置光标</span>
<span class="n">scrollbar_position</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">disabled</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">禁用滚动条</span>
<span class="n">titlebars</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">no</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">禁用标题栏</span>
<span class="n">background_darkness</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mf">0.4</span>
<span class="n">background_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">transparent</span><span class="w"> </span><span class="p">#</span><span class="w"> </span><span class="err">背景类型可以设置为图片</span>
</code></pre></div>
<p>更多配置可以参见配置文件:</p>
<div class="highlight"><pre><span></span><code>man<span class="w"> </span>terminator_config
</code></pre></div>
<h2 id="3-guake">3 Guake</h2>
<p><code>Guake</code>是一个下拉式的gnome桌面环境下的终端程序,因此你只需要按一个键F12就可以调用他,然后再按一次以便隐藏他。Guake支持<code>快捷键</code>、<code>标签</code>、<code>背景透明</code>等特性。</p>
<h3 id="31-guake">3.1 安装 Guake</h3>
<div class="highlight"><pre><span></span><code>sudo apt-get install guake
</code></pre></div>
<h3 id="32-guake">3.2 使用Guake</h3>
<p>启用guake后即可按<code>F12</code>调出来,很酷
快捷键和gnome terminal相同</p>
<div class="highlight"><pre><span></span><code>Ctrl-T 新建标签
F2 重命名标签
Ctrl-PageUp 上一个标签
Ctrl-PageDwon 下一个标签
F11 全屏
F12 隐藏/显示
</code></pre></div>用bottle+mongodb写的blog程序支持mysql啦2012-08-14T09:16:00+08:002012-08-14T09:16:00+08:00coldtag:www.linuxzen.com,2012-08-14:/yong-bottlemongodbxie-de-blogcheng-xu-zhi-chi-mysqlla.html<p>前面博文提到过,本人用bottle+mongodb实现了以blog程序,</p>
<p>最近有些空闲时间,就重新用mvc的模式重写了一下,因为mongodb太过耗费内存,对一些小型的vps太过吃力所以加入了mysql的支持,</p>
<p>虽然较上次有些完善,但是还是有很 …</p><p>前面博文提到过,本人用bottle+mongodb实现了以blog程序,</p>
<p>最近有些空闲时间,就重新用mvc的模式重写了一下,因为mongodb太过耗费内存,对一些小型的vps太过吃力所以加入了mysql的支持,</p>
<p>虽然较上次有些完善,但是还是有很多不足代码放在了googlecode上,由于最近一直在使用git,索性也就新建了一个git的项目,之前svn的项目也会更新.</p>
<p>所以大家想浏览代码可以到下面两个地方去,有什么不足和建议还请指教,如果你也想加入进来,可以留言或发email给我:</p>
<p>git:<a href="http://code.google.com/p/linuxzen/source/browse/water">http://code.google.com/p/linuxzen/source/browse/water</a></p>
<p>svn:http://code.google.com/p/sharepythoncode/source/browse/water/</p>python里的三目运算2012-08-10T16:28:00+08:002012-08-10T16:28:00+08:00coldtag:www.linuxzen.com,2012-08-10:/pythonli-de-san-mu-yun-suan.html<p>下面说的和三目运算有点相似,但又不一样,实在不知道该如何拟定标题,先就是这个标题吧,大家都知道python中没有三目运算,但是<code>and</code>/<code>or</code>有点 …</p><p>下面说的和三目运算有点相似,但又不一样,实在不知道该如何拟定标题,先就是这个标题吧,大家都知道python中没有三目运算,但是<code>and</code>/<code>or</code>有点类似三目运算:</p>
<h2 id="andor">and/or</h2>
<p>单独使用表示逻辑关系与和或,也可以组和使用,用法如下</p>
<h3 id="and">and</h3>
<p>and前后如果某一个值为假(False, '', [], {}, None…)则返回第一个假值
如果所有值都为真则返回最后一个真值</p>
<h3 id="or">or</h3>
<p>如果or任意一个值为真,则立刻返回这个值
如果所有值都为假,则or返回最后一个假值</p>
<h3 id="_1">例子</h3>
<div class="highlight"><pre><span></span><code><span class="n">result</span> <span class="o">=</span> <span class="s1">'test'</span> <span class="ow">and</span> <span class="kc">True</span> <span class="c1"># result = True</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">'test'</span> <span class="ow">and</span> <span class="s1">'ortest'</span> <span class="c1"># result = ortest</span>
<span class="n">result</span> <span class="o">=</span> <span class="kc">False</span> <span class="ow">and</span> <span class="s1">'ortest'</span> <span class="c1"># result = False</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">''</span> <span class="ow">and</span> <span class="kc">None</span> <span class="c1"># result = ''</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">''</span> <span class="ow">or</span> <span class="s2">"Hall"</span> <span class="c1"># result = Hall</span>
<span class="n">result</span> <span class="o">=</span> <span class="kc">False</span> <span class="ow">or</span> <span class="kc">None</span> <span class="c1"># result = None</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">'test'</span> <span class="ow">or</span> <span class="s1">'nottest'</span> <span class="c1"># result = test</span>
</code></pre></div>
<h2 id="if-else">使用单行if else 模拟三目运算</h2>
<p>result if True / False else fresult
if为真时候结果为result,为假的时候结果为fresult</p>
<div class="highlight"><pre><span></span><code><span class="n">result</span> <span class="o">=</span> <span class="s1">'test'</span> <span class="k">if</span> <span class="kc">True</span> <span class="k">else</span> <span class="s1">'not test'</span> <span class="c1"># result = 'test'</span>
<span class="n">result</span> <span class="o">=</span> <span class="s1">'test'</span> <span class="k">if</span> <span class="kc">False</span> <span class="k">else</span> <span class="s1">'not test'</span> <span class="c1"># result = 'not test'</span>
</code></pre></div>Python 断点调试2012-08-10T16:16:00+08:002012-08-10T16:16:00+08:00coldtag:www.linuxzen.com,2012-08-10:/python-duan-dian-diao-shi.html<h2 id="pdb">pdb模块</h2>
<p>pdb是一个Python 内置的调式模块这里用来介绍用它进行断点调试</p>
<h3 id="_1">插入断点</h3>
<p>在需要插入断点的地方插入如下代码可以插入一个断 …</p><h2 id="pdb">pdb模块</h2>
<p>pdb是一个Python 内置的调式模块这里用来介绍用它进行断点调试</p>
<h3 id="_1">插入断点</h3>
<p>在需要插入断点的地方插入如下代码可以插入一个断点
import pdb; pdb.set_trace()
当Python执行到这条语句时在运行shell里就会中断执行出现一个类似下面的shell窗口</p>
<div class="highlight"><pre><span></span><code><span class="o">></span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="err">这里会出现当前运行程序的信息</span><span class="p">,</span><span class="err">源文件和当前函数</span>
<span class="o">-></span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="err">这里是将要运行的语句</span>
<span class="w"> </span><span class="p">(</span><span class="n">Pdb</span><span class="p">)</span>
</code></pre></div>
<h3 id="pdb_1">pdb指令</h3>
<p>进入(Pdb)后有很多命令可以使用,可以使用 <code>h</code> 查看帮助</p>
<ul>
<li>l 查看代码上下文</li>
<li>p var 监视变量var</li>
<li>n 单步执行</li>
<li>b line 在line行插入断点</li>
<li>c 继续到下一个断点,没有则执行程序</li>
<li>r 执行到函数返回前</li>
</ul>ssh证书登录错误2012-08-10T16:11:00+08:002012-08-10T16:11:00+08:00coldtag:www.linuxzen.com,2012-08-10:/sshzheng-shu-deng-lu-cuo-wu.html<h2>错误描述</h2>
<p>使用证书ssh链接的时候提示下面错误信息</p>
<h2><code>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).</code>
可能原因</h2>
<p>authorizedkeys 或.ssh的权限太open .ssh 目录改成755 权限 authorizedkeys 改成600</p>
<h2>解决</h2>
<p>查看日志:
<code>cat /var/log/secure</code>
发现
<code>Aug 8 17:15 …</code></p><h2>错误描述</h2>
<p>使用证书ssh链接的时候提示下面错误信息</p>
<h2><code>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).</code>
可能原因</h2>
<p>authorizedkeys 或.ssh的权限太open .ssh 目录改成755 权限 authorizedkeys 改成600</p>
<h2>解决</h2>
<p>查看日志:
<code>cat /var/log/secure</code>
发现
<code>Aug 8 17:15:13 CentOS62 sshd[5624]: Authentication refused: bad ownership or modes for file /home/abc/.ssh/authorized_keys</code>
查看.ssh权限为775
.ssh 手动创建的时候是775权限,改成755权限后正常
<code># chmod 755 ~/.ssh</code></p>lambda 结合map/filter/reduce/sorted等函数对列表进行高效操作2012-08-10T16:02:00+08:002012-08-10T16:02:00+08:00coldtag:www.linuxzen.com,2012-08-10:/lambda-jie-he-mapfilterreducesorteddeng-han-shu-dui-lie-biao-jin-xing-gao-xiao-cao-zuo.html<p>lambda 结合map/filter/reduce/sorted等函数对列表进行高效操作</p>
<h2 id="_1">介绍</h2>
<h3 id="lambda">lambda</h3>
<p>Python用于支持将函数赋值给变量的一个操作符
默认是返回的,所以不用再加return关键字,不然会报错</p>
<div class="highlight"><pre><span></span><code><span class="n">result</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span>
<span class="n">result</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># return 4</span>
<span class="nb">map</span><span class="p">()</span><span class="o">/</span><span class="nb">filter</span><span class="p">()</span><span class="o">/</span><span class="n">reduce</span><span class="p">()</span>
</code></pre></div>
<p>需要两个参数,第一个 …</p><p>lambda 结合map/filter/reduce/sorted等函数对列表进行高效操作</p>
<h2 id="_1">介绍</h2>
<h3 id="lambda">lambda</h3>
<p>Python用于支持将函数赋值给变量的一个操作符
默认是返回的,所以不用再加return关键字,不然会报错</p>
<div class="highlight"><pre><span></span><code><span class="n">result</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span>
<span class="n">result</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># return 4</span>
<span class="nb">map</span><span class="p">()</span><span class="o">/</span><span class="nb">filter</span><span class="p">()</span><span class="o">/</span><span class="n">reduce</span><span class="p">()</span>
</code></pre></div>
<p>需要两个参数,第一个是一个处理函数,第二个是一个序列(list,tuple,dict)</p>
<h3 id="map">map()</h3>
<p>将序列中的元素通过处理函数处理后返回一个新的列表</p>
<h3 id="filter">filter()</h3>
<p>将序列中的元素通过函数过滤后返回一个新的列表</p>
<h3 id="reduce">reduce()</h3>
<p>将序列中的元素通过一个二元函数处理返回一个结果</p>
<h2 id="lambda_1">将上面三个函数和lambda结合使用</h2>
<div class="highlight"><pre><span></span><code><span class="n">li</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">]</span>
<span class="c1"># 序列中的每个元素加1</span>
<span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">li</span><span class="p">)</span> <span class="c1"># [2,3,4,5,6]</span>
<span class="c1"># 返回序列中的偶数</span>
<span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span><span class="p">,</span> <span class="n">li</span><span class="p">)</span> <span class="c1"># [2, 4]</span>
<span class="c1"># 返回所有元素相乘的结果</span>
<span class="n">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">y</span><span class="p">,</span> <span class="n">li</span><span class="p">)</span> <span class="c1"># 1*2*3*4*5 = 120</span>
</code></pre></div>
<h2 id="sorted-lambda">sorted() 结合lambda对列表进行排序</h2>
<p>sorted 用于列表的排序,比列表自带的更加智能
有两个列表,每个列表中都有一个字典([{},{}])要求将两个这样的列表合并后按照时间排序,
两个列表中的时间为了能够通过json输出已经由时间格式转变为字符串格式.字段名为 sort_time
现在将他们按照倒序排列</p>
<h3 id="sorted">sorted 的用法</h3>
<p>sorted(iterable, cmp=None, key=None, reverse=False) --> new sorted list
* terable:是可迭代类型;
* cmp:用于比较的函数,比较什么由key决定,有默认值,迭代集合中的一项;
* key:用列表元素的某个属性和函数进行作为关键字,有默认值,迭代集合中的一项;
* reverse:排序规则. reverse = True 或者 reverse = False,有默认值。
* 返回值:是一个经过排序的可迭代类型,与iterable一样。</p>
<h3 id="sortedlambdasort_time">sorted()结合lambda对可迭代类型用sort_time排序</h3>
<div class="highlight"><pre><span></span><code><span class="nb">sorted</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">d</span><span class="p">:</span> <span class="n">d</span><span class="p">[</span><span class="s1">'sort_time'</span><span class="p">],</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>Python 常用的列表操作2012-08-10T15:48:00+08:002012-08-10T15:48:00+08:00coldtag:www.linuxzen.com,2012-08-10:/python-chang-yong-de-lie-biao-cao-zuo.html<p>这里介绍几个常用的列表操作</p>
<h3 id="_1">添加元素</h3>
<p>添加元素使用列表的内置方法append</p>
<div class="highlight"><pre><span></span><code><span class="n">number</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">number</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># number = [1, 2, 3, 4, 5]</span>
<span class="n">number</span><span class="o">.</span><span class="n">append</span><span class="p">([</span><span class="mi">6</span><span class="p">,</span><span class="mi">7</span><span class="p">])</span> <span class="c1"># number …</span></code></pre></div><p>这里介绍几个常用的列表操作</p>
<h3 id="_1">添加元素</h3>
<p>添加元素使用列表的内置方法append</p>
<div class="highlight"><pre><span></span><code><span class="n">number</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">number</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># number = [1, 2, 3, 4, 5]</span>
<span class="n">number</span><span class="o">.</span><span class="n">append</span><span class="p">([</span><span class="mi">6</span><span class="p">,</span><span class="mi">7</span><span class="p">])</span> <span class="c1"># number = [1, 2, 3, 4, 5, [6, 7]]</span>
<span class="n">number</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s1">'a'</span><span class="p">:</span><span class="s1">'b'</span><span class="p">})</span> <span class="c1"># number = [1, 2, 3, 4, [6, 7], {'a', :'b'}</span>
</code></pre></div>
<p>可以看到强大的python列表可以嵌套任意类型</p>
<h3 id="_2">列表相加</h3>
<p>要想连接两个列表,可以使用+号连接</p>
<div class="highlight"><pre><span></span><code><span class="n">a</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="n">b</span> <span class="o">=</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">]</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span> <span class="c1"># c = [1, 2, 3, 4, 5, 6]</span>
</code></pre></div>
<p>也可以使用列表内置方法extend连接两个列表</p>
<div class="highlight"><pre><span></span><code><span class="n">a</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="n">b</span> <span class="o">=</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">]</span>
<span class="n">a</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="c1"># a = [1, 2, 3, 4, 5, 6]</span>
</code></pre></div>
<p>用+号会创建一个新通对象,使用extend则在原来的对象上面修改</p>
<h3 id="_3">列表去重复</h3>
<p>列表本身没有去除重复的功能,但是可以借助python的另外一个类型set(help(set)查看)</p>
<div class="highlight"><pre><span></span><code><span class="n">a</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
<span class="n">b</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">a</span><span class="p">))</span> <span class="c1"># b = [1, 2, 3]</span>
</code></pre></div>
<p>也可以借助字典类型的内置方法</p>
<div class="highlight"><pre><span></span><code><span class="n">a</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="n">b</span> <span class="o">=</span> <span class="p">{}</span><span class="o">.</span><span class="n">fromkeys</span><span class="p">(</span><span class="n">a</span><span class="p">)</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="c1"># b = [1, 2, 3]</span>
</code></pre></div>Python超简单截取中文字符串2012-07-11T17:55:00+08:002012-07-11T17:55:00+08:00coldtag:www.linuxzen.com,2012-07-11:/pythonchao-jian-dan-jie-qu-zhong-wen-zi-fu-chuan.html<p>web应用难免会截取字符串的需求,Python中截取英文很容易:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">s</span> <span class="o">=</span> <span class="s1">'abce'</span>
<span class="o">>>></span> <span class="n">s</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span>
<span class="s1">'abc'</span>
</code></pre></div>
<p>但是截取utf-8的中文机会截取一半导致一些不是乱码的乱码.其实utf8截取很简单,这里记下来作为备忘</p>
<div class="highlight"><pre><span></span><code><span class="c1">#-*- coding:utf8 -*-</span>
<span class="n">s</span> <span class="o">=</span> <span class="sa">u</span><span class="s1">'中文截取'</span>
<span class="n">s</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf8'</span><span class="p">)[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf8'</span><span class="p">)</span>
<span class="c1"># 结果u'中文截取</span>
</code></pre></div>
<p>就这么 …</p><p>web应用难免会截取字符串的需求,Python中截取英文很容易:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">s</span> <span class="o">=</span> <span class="s1">'abce'</span>
<span class="o">>>></span> <span class="n">s</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span>
<span class="s1">'abc'</span>
</code></pre></div>
<p>但是截取utf-8的中文机会截取一半导致一些不是乱码的乱码.其实utf8截取很简单,这里记下来作为备忘</p>
<div class="highlight"><pre><span></span><code><span class="c1">#-*- coding:utf8 -*-</span>
<span class="n">s</span> <span class="o">=</span> <span class="sa">u</span><span class="s1">'中文截取'</span>
<span class="n">s</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf8'</span><span class="p">)[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf8'</span><span class="p">)</span>
<span class="c1"># 结果u'中文截取</span>
</code></pre></div>
<p>就这么简单</p>mongoengine使用笔记2012-06-23T10:02:00+08:002012-06-23T10:02:00+08:00coldtag:www.linuxzen.com,2012-06-23:/mongoengineshi-yong-bi-ji.html<p>最近重新拾起Django,但是Django并不支持mongodb,但是有一个模块mongoengine可以实现Django Model类似的封装.但是mongoengine的中文文档几乎没有,有的也是简短的几句介绍和使用.下面我就分享一下我在使用过程 …</p><p>最近重新拾起Django,但是Django并不支持mongodb,但是有一个模块mongoengine可以实现Django Model类似的封装.但是mongoengine的中文文档几乎没有,有的也是简短的几句介绍和使用.下面我就分享一下我在使用过程中所记录下的一些笔记,可能有点乱.大家可以参考一下.</p>
<h2 id="mongoengine">安装mongoengine</h2>
<div class="highlight"><pre><span></span><code><span class="n">easy_install</span> <span class="n">pymongo</span> <span class="c1"># 依赖库</span>
<span class="n">easy_install</span> <span class="n">mongoengine</span>
</code></pre></div>
<h2 id="_1">基本使用</h2>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">mongoengine</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="c1"># 连接数据库</span>
<span class="n">connect</span><span class="p">(</span><span class="s1">'blog'</span><span class="p">)</span> <span class="c1"># 连接本地blog数据库</span>
<span class="c1"># 如需验证和指定主机名</span>
<span class="c1"># connect('blog', host='192.168.3.1', username='root', password='1234')</span>
<span class="c1"># 定义分类文档</span>
<span class="k">class</span> <span class="nc">Categories</span><span class="p">(</span><span class="n">Document</span><span class="p">):</span>
<span class="s1">' 继承Document类,为普通文档 '</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">artnum</span> <span class="o">=</span> <span class="n">IntField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">DateTimeField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">(),</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>
<p>和Django的model使用很类似,所以也不解释什么.</p>
<h3 id="_2">插入</h3>
<div class="highlight"><pre><span></span><code><span class="n">cate</span> <span class="o">=</span> <span class="n">Categories</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Linux"</span><span class="p">)</span> <span class="c1"># 如果required为True则必须赋予初始值,如果有default,赋予初始值则使用默认值</span>
<span class="n">cate</span><span class="o">.</span><span class="n">save</span><span class="p">()</span> <span class="c1"># 保存到数据库</span>
</code></pre></div>
<h3 id="_3">查询和更新</h3>
<p>文档类有一个 objects 属性.我们使用它来查询数据库.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 返回集合里的所有文档对象的列表</span>
<span class="n">cate</span> <span class="o">=</span> <span class="n">Categories</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
<span class="c1"># 返回所有符合查询条件的结果的文档对象列表</span>
<span class="n">cate</span> <span class="o">=</span> <span class="n">Categories</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Python"</span><span class="p">)</span>
<span class="c1"># 更新查询到的文档:</span>
<span class="n">cate</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="s2">"LinuxZen"</span>
<span class="n">cate</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>
<span class="n">查询数组</span> <span class="n">默认查询数组</span><span class="s2">"="</span><span class="n">代表的意思是in</span><span class="p">:</span>
<span class="k">class</span> <span class="nc">Posts</span><span class="p">(</span><span class="n">Document</span><span class="p">):</span>
<span class="n">artid</span> <span class="o">=</span> <span class="n">IntField</span><span class="p">(</span><span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">title</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">author</span> <span class="o">=</span> <span class="n">ReferenceField</span><span class="p">(</span><span class="n">User</span><span class="p">)</span>
<span class="n">tags</span> <span class="o">=</span> <span class="n">ListField</span><span class="p">(</span><span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">20</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">categories</span> <span class="o">=</span> <span class="n">ReferenceField</span><span class="p">(</span><span class="n">Categories</span><span class="p">),</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">comments</span> <span class="o">=</span> <span class="n">IntField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="c1"># 将会返回所有tags包含coding的文档</span>
<span class="n">Posts</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">tags</span><span class="o">=</span><span class="s1">'coding'</span><span class="p">)</span>
</code></pre></div>
<h3 id="referencefield">ReferenceField 引用字段:</h3>
<p>通过引用字段可以通过文档直接获取引用字段引用的那个文档:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Categories</span><span class="p">(</span><span class="n">Document</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">artnum</span> <span class="o">=</span> <span class="n">IntField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">DateTimeField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">(),</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Posts</span><span class="p">(</span><span class="n">Document</span><span class="p">):</span>
<span class="n">title</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">tags</span> <span class="o">=</span> <span class="n">ListField</span><span class="p">(</span><span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">20</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">categories</span> <span class="o">=</span> <span class="n">ReferenceField</span><span class="p">(</span><span class="n">Categories</span><span class="p">)</span>
</code></pre></div>
<h4 id="_4">插入引用字段</h4>
<div class="highlight"><pre><span></span><code><span class="n">cate</span> <span class="o">=</span><span class="n">Categories</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Linux"</span><span class="p">)</span>
<span class="n">cate</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="n">post</span> <span class="o">=</span> <span class="n">Posts</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">"Linuxzen.com"</span><span class="p">,</span> <span class="n">content</span><span class="o">=</span><span class="s2">"Linuxzen.com"</span><span class="p">,</span><span class="n">tags</span><span class="o">=</span><span class="p">[</span><span class="s2">"Linux"</span><span class="p">,</span><span class="s2">"web"</span><span class="p">],</span> <span class="n">categories</span><span class="o">=</span><span class="n">cate</span><span class="p">)</span>
<span class="n">post</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
</code></pre></div>
<h4 id="_5">通过引用字段直接获取引用文档对象</h4>
<p>一般文档查询会返回一个列表(尽管只有一个结果),我们想要获得一个文档对象可以使用索引获取第一个文档对象,但是mongoengine建议使用first()来获取第一个:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">cate</span> <span class="o">=</span> <span class="n">Posts</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">all</span><span class="p">()</span><span class="o">.</span><span class="n">first</span><span class="p">()</span><span class="o">.</span><span class="n">categories</span>
<span class="o">>>></span> <span class="n">cate</span>
<span class="o">>>></span> <span class="n">cate</span><span class="o">.</span><span class="n">name</span>
<span class="sa">u</span><span class="s1">'Linux'</span>
</code></pre></div>
<p>查询包含Linux分类的文章</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">cate</span> <span class="o">=</span> <span class="n">Categories</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Linux"</span><span class="p">)</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">Posts</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">categories</span><span class="o">=</span><span class="n">cate</span><span class="p">)</span>
</code></pre></div>
<h3 id="embeddeddocument">EmbeddedDocument 嵌入文档</h3>
<p>继承EmbeddedDocument的文档类就是嵌入文档,嵌入文档用于嵌入其他文档的EmbeddedDocumentField 字段,比如上面例子的tags字段如果改成嵌入文档的话可以将Posts文档类改成如下方式:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Posts</span><span class="p">(</span><span class="n">Document</span><span class="p">):</span>
<span class="n">title</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">content</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">(</span><span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">tags</span> <span class="o">=</span> <span class="n">ListField</span><span class="p">(</span><span class="n">EmbeddedDocumentField</span><span class="p">(</span><span class="s1">'Tags'</span><span class="p">)</span><span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">categories</span> <span class="o">=</span> <span class="n">ReferenceField</span><span class="p">(</span><span class="n">Categories</span><span class="p">)</span>
</code></pre></div>
<p>还需要添加一个Tags嵌入文档类:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Tags</span><span class="p">(</span><span class="n">EmbeddedDocument</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringField</span><span class="p">()</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">DateTimeField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">())</span>
</code></pre></div>
<p>我们像如下方式插入Posts文档中的Tags</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">tag</span> <span class="o">=</span> <span class="n">Tags</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Linuxzen"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">post</span> <span class="o">=</span> <span class="n">Posts</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">"Linuxzen.com"</span><span class="p">,</span> <span class="n">content</span><span class="o">=</span><span class="s2">"Linuxzen.com"</span><span class="p">,</span> <span class="n">tags</span><span class="o">=</span><span class="p">[</span><span class="n">tag</span><span class="p">],</span> <span class="n">categories</span><span class="o">=</span><span class="n">cate</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">tag</span> <span class="o">=</span> <span class="n">Tags</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"mysite"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">post</span><span class="o">.</span><span class="n">tags</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">tag</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">post</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">tags</span> <span class="o">=</span> <span class="n">post</span><span class="o">.</span><span class="n">tags</span>
<span class="o">>>></span> <span class="k">for</span> <span class="n">tag</span> <span class="ow">in</span> <span class="n">tags</span><span class="p">:</span>
<span class="nb">print</span> <span class="n">tag</span><span class="o">.</span><span class="n">name</span>
<span class="n">Linuxzen</span>
<span class="n">mysite</span>
</code></pre></div>
<h3 id="_6">时间段查询</h3>
<div class="highlight"><pre><span></span><code> <span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">year</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">month</span><span class="p">),</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">int</span><span class="p">(</span><span class="n">month</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">></span> <span class="mi">12</span><span class="p">:</span>
<span class="n">emonth</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">eyear</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">year</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">emonth</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">month</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">eyear</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">year</span><span class="p">)</span>
<span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="n">eyear</span><span class="p">,</span> <span class="n">emonth</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">articles</span> <span class="o">=</span> <span class="n">Posts</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">date__gte</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">date__lt</span><span class="o">=</span><span class="n">end</span><span class="p">)</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="s1">'-date'</span><span class="p">)</span>
</code></pre></div>
<h3 id="_7">分片</h3>
<p>slice用于分片</p>
<div class="highlight"><pre><span></span><code><span class="c1"># comments - skip 5, limit 10</span>
<span class="n">Page</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">fields</span><span class="p">(</span><span class="n">slice__comments</span><span class="o">=</span><span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">])</span>
<span class="c1"># 也可以使用索引值分片</span>
<span class="c1"># limit 5</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="p">[:</span><span class="mi">5</span><span class="p">]</span>
<span class="c1"># skip 5</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="p">[</span><span class="mi">5</span><span class="p">:]</span>
<span class="c1"># skip 10, limit 15</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="p">[</span><span class="mi">10</span><span class="p">:</span><span class="mi">15</span><span class="p">]</span>
</code></pre></div>
<h3 id="_8">使用原始语句查询</h3>
<p>如果想使用原始的pymongo查询方式可以使用__raw__操作符 Page.objects(<strong>raw</strong>={'tags':'coding'})
使用$inc和$set操作符</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 更新嵌入文档comments字段by的值为joe的文档字段votes增加1</span>
<span class="n">Page</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">comments_by</span><span class="o">=</span><span class="s2">"joe"</span><span class="p">)</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">inc__votes</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="c1"># 更新嵌入文档comments字段by的值为joe的文档字段votes设置为1</span>
<span class="n">Page</span><span class="o">.</span><span class="n">objects</span><span class="p">(</span><span class="n">comments_by</span><span class="o">=</span><span class="s2">"joe"</span><span class="p">)</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">set__votes</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</code></pre></div>
<h3 id="_9">其他技巧</h3>
<div class="highlight"><pre><span></span><code><span class="c1">#查询结果转换成字典</span>
<span class="n">users_dict</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="p">()</span><span class="o">.</span><span class="n">to_mongo</span><span class="p">()</span>
<span class="c1"># 排序,按日期排列</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="s2">"date"</span><span class="p">)</span>
<span class="c1"># 按日期倒序</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="s2">"-date"</span><span class="p">)</span>
</code></pre></div>bottle使用Python装饰器巧妙解决用户验证2012-06-16T17:33:00+08:002012-06-16T17:33:00+08:00coldtag:www.linuxzen.com,2012-06-16:/bottleshi-yong-pythonzhuang-shi-qi-qiao-miao-jie-jue-yong-hu-yan-zheng.html<p>上篇文章发布了一个自己写的用bottle写的web程序,其中收获最大就是Python装饰器的使用.前几天也是比较忙,所以没能分享出来,今天就给大家分享一下 …</p><p>上篇文章发布了一个自己写的用bottle写的web程序,其中收获最大就是Python装饰器的使用.前几天也是比较忙,所以没能分享出来,今天就给大家分享一下.</p>
<p>首先来分析下需求,web程序后台需要认证,后台页面包含多个页面,最普通的方法就是为每个url添加认证,但是这样就需要每个每个绑定url的后台函数都需要添加类似或者相同的代码,但是这样做代码就过度冗余,而且不利于扩展.</p>
<p>接下来我们先不谈及装饰器,我们都知道Python是个很强大的语言,她可以将函数当做参数传递给函数,最简单的:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">p</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'Hello,world'</span>
<span class="k">def</span> <span class="nf">funcfactor</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'calling function named'</span><span class="p">,</span> <span class="n">func</span><span class="o">.</span><span class="vm">__name__</span>
<span class="n">func</span><span class="p">()</span>
<span class="nb">print</span> <span class="s1">'end'</span>
<span class="n">funcfactor</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
<span class="c1"># 输出为:</span>
<span class="c1"># calling function named p</span>
<span class="c1"># Hello,world</span>
<span class="c1"># end</span>
</code></pre></div>
<p>一目了然的程序,定义一个函数p(),将函数p当做参数传递给喊出funcfactor,在执行p函数前后加上一些动作.</p>
<p>我们还可以这么做:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">p</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'Hello,world'</span>
<span class="k">def</span> <span class="nf">funcfactor</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'calling function named'</span><span class="p">,</span> <span class="n">func</span><span class="o">.</span><span class="vm">__name__</span>
<span class="k">return</span> <span class="n">func</span>
<span class="n">func</span> <span class="o">=</span> <span class="n">funcfactor</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
<span class="n">func</span><span class="p">()</span>
<span class="c1"># 输出为:</span>
<span class="c1"># calling function named p</span>
<span class="n">Hello</span><span class="p">,</span><span class="n">world</span>
</code></pre></div>
<p>正如你看到的,我们可以将函数返回然后赋予一个变量,留待稍后调用.但是这种情况下我们要想在函数执行后做点什么就不可能,但是我们的Python是强大的,Python可以在函数中再嵌套一个函数,我们可以像下面这么做:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">p</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'Hello, world'</span>
<span class="k">def</span> <span class="nf">funcfactor</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'do something at start'</span>
<span class="n">func</span><span class="p">()</span>
<span class="nb">print</span> <span class="s1">'do something at end'</span>
<span class="k">return</span> <span class="n">wrapper</span>
<span class="n">func</span> <span class="o">=</span> <span class="n">funcfactor</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
<span class="n">func</span><span class="p">()</span>
<span class="c1">#输出为:</span>
<span class="c1"># do something at start</span>
<span class="c1"># Hello, world</span>
<span class="c1"># do something at end</span>
</code></pre></div>
<p>下面我们来看看装饰器,上面的代码虽然实现的一个很困难的任务,但是还不够优雅,而且代码不符合Python的哲学思想,所以装饰器就应声而出,装饰器没有和上面的原理相同,同样用于包装函数,只是代码实现上更加优雅和便于阅读.装饰器以@开头后面跟上装饰器的名称,紧接着下一行就是要包装的函数体,上面的例子用装饰器可用如下方式实现:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'do something at start'</span>
<span class="n">func</span><span class="p">()</span>
<span class="nb">print</span> <span class="s1">'do something at end'</span>
<span class="k">return</span> <span class="n">wrapper</span>
<span class="nd">@decorator</span>
<span class="k">def</span> <span class="nf">p</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'Hello, world'</span>
<span class="n">p</span><span class="p">()</span>
<span class="c1">#输出为:</span>
<span class="c1"># do something at start</span>
<span class="c1"># Hello, world</span>
<span class="c1"># do something at end</span>
</code></pre></div>
<p>实际上装饰器并没有性能方面或其他方面的提升,仅仅是一种语法糖,就是上面一个例子的改写,这样更加优雅和便与阅读.
如果我们的p()函数不想仅仅只输Hello,world,我们想向某些我们指定的人打招呼:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kargs</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'do something at start'</span>
<span class="n">func</span><span class="p">(</span><span class="o">**</span><span class="n">kargs</span><span class="p">)</span>
<span class="nb">print</span> <span class="s1">'do something at end'</span>
<span class="k">return</span> <span class="n">wrapper</span>
<span class="nd">@decorator</span>
<span class="k">def</span> <span class="nf">p</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'Hello'</span><span class="p">,</span> <span class="n">name</span>
<span class="n">p</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"Jim"</span><span class="p">)</span>
<span class="c1">#输出为:</span>
<span class="c1"># do something at start</span>
<span class="c1"># Hello Jim</span>
<span class="c1"># do something at end</span>
</code></pre></div>
<p>装饰器在装饰不需要参数的装饰器嵌套函数不是必须得,如果被装饰的函数需要参数,必须嵌套一个函数来处理参数.
写到这里想必大家也知道装饰器的用法和作用.现在回到正题,如何优雅的给后台url加上验证功能?毫无疑问我们使用装饰器来处理:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">blog_auth</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="w"> </span><span class="sd">'''</span>
<span class="sd"> 定义一个装饰器用于装饰需要验证的页面</span>
<span class="sd"> 装饰器必须放在route装饰器下面</span>
<span class="sd"> '''</span>
<span class="c1"># 定义包装函数</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kargs</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># 读取cookie</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
<span class="n">shell</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'shell'</span><span class="p">]</span>
<span class="k">except</span><span class="p">:</span>
<span class="c1"># 出现异常则重定向到登录页面</span>
<span class="n">redirect</span><span class="p">(</span><span class="s1">'/login'</span><span class="p">)</span>
<span class="c1"># 验证用户数据</span>
<span class="k">if</span> <span class="n">checkShell</span><span class="p">(</span><span class="n">user</span><span class="p">,</span> <span class="n">shell</span><span class="p">):</span>
<span class="c1"># 校验成功则返回函数</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">**</span><span class="n">kargs</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># 否则则重定向到登录页面</span>
<span class="n">redirect</span><span class="p">(</span><span class="s1">'/login'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">wrapper</span>
</code></pre></div>
<p>可以再需要验证的地方添加blog_auth装饰器:</p>
<div class="highlight"><pre><span></span><code><span class="nd">@route</span><span class="p">(</span><span class="s1">'/admin:#/?#'</span><span class="p">)</span>
<span class="nd">@blog_auth</span>
<span class="k">def</span> <span class="nf">admin</span><span class="p">():</span>
<span class="w"> </span><span class="sd">'''</span>
<span class="sd"> 用于显示后台管理首页</span>
<span class="sd"> '''</span>
<span class="n">TEMPLATE</span><span class="p">[</span><span class="s1">'title'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'仪表盘 | '</span> <span class="o">+</span> <span class="n">TEMPLATE</span><span class="p">[</span><span class="s1">'BLOG_NAME'</span><span class="p">]</span>
<span class="n">TEMPLATE</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
<span class="n">articles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">article</span> <span class="ow">in</span> <span class="n">db</span><span class="o">.</span><span class="n">posts</span><span class="o">.</span><span class="n">find</span><span class="p">()</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="s2">"date"</span><span class="p">,</span><span class="n">DESCENDING</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">articles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">article</span><span class="p">)</span>
<span class="c1"># 将文章列表交给前台模版</span>
<span class="n">TEMPLATE</span><span class="p">[</span><span class="s1">'articles'</span><span class="p">]</span> <span class="o">=</span> <span class="n">articles</span>
<span class="k">return</span> <span class="n">template</span><span class="p">(</span><span class="s1">'admin.html'</span><span class="p">,</span><span class="n">TEMPLATE</span><span class="p">)</span>
</code></pre></div>
<p>至此bottle验证的问题就很优雅的用装饰器解决了.</p>用bottle+mongodb写的一个blog程序2012-06-16T16:20:00+08:002012-06-16T16:20:00+08:00coldtag:www.linuxzen.com,2012-06-16:/yong-bottlemongodbxie-de-yi-ge-blogcheng-xu.html<p>我个人觉得更好更快的学习和掌握某个东西最好的方法就是使用它,多使用它.然后在一次次的解决问题中来快速掌握 …</p><p>我个人觉得更好更快的学习和掌握某个东西最好的方法就是使用它,多使用它.然后在一次次的解决问题中来快速掌握和了解它.你觉得呢?前段时间接触了bottle这个轻量web框架,和nosql数据库mongodb,为了掌握和了解这她们,我自己做了一个blog程序,参照了vimer.cn里的设计的物理设计.是用bottle作为web开发框架,mongodb作为后台数据库.主要实现功能:</p>
<p>前台显示文章:
按分类显示
按标签显示
按月份归档显示
最新文章
评论
后台管理:
管理文章
管理分类
管理评论
发表文章
用户验证</p>
<p>目录结构</p>
<p>water</p>
<p>+-app 程序目录</p>
<p>|----admin.py 后台管理</p>
<p>|----blog.py 前台显示</p>
<p>|----dbconn.py 数据库连接</p>
<p>|----encrypt.py包含加密函数</p>
<p>+-static 静态文件目录:包括js css image</p>
<p>+-views 模版目录</p>
<p>+-index.py 用于启动整个程序</p>
<p>+-initiate.py 初始化脚本,用于创建一个管理用户</p>
<p>+-setting.py 设置文件,各种设置</p>
<p>+-static.py 用于程序处理静态文件</p>
<p>项目代码放在了google code上,可以访问下面链接浏览:
http://code.google.com/p/sharepythoncode/source/browse/water/</p>bottle的cookie操作小记(获取不是在当前页面创建的cookie)2012-06-02T17:51:00+08:002012-06-02T17:51:00+08:00coldtag:www.linuxzen.com,2012-06-02:/bottlede-cookiecao-zuo-xiao-ji-huo-qu-bu-shi-zai-dang-qian-ye-mian-chuang-jian-de-cookie.html<p>这两天为用bottle+mongodb写的一个项目加上登录功能,无奈怎么都获取不到保存的cookie,文档给出让我们这样操作cookie的代码片段:</p>
<div class="highlight"><pre><span></span><code><span class="nd">@route</span><span class="p">(</span><span class="err">’</span><span class="o">/</span><span class="n">login</span><span class="err">’</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">login</span> <span class="p">():</span>
<span class="n">username</span> <span class="o">=</span> <span class="n">request</span> <span class="o">.</span><span class="n">forms</span> <span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="err">’</span><span class="n">username</span> <span class="err">’</span><span class="p">)</span>
<span class="n">password</span> <span class="o">=</span> <span class="n">request</span> <span class="o">.</span><span class="n">forms</span> <span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="err">’</span><span class="n">password</span> <span class="err">’</span><span class="p">)</span>
<span class="k">if</span> <span class="n">check_user_credentials</span><span class="p">(</span><span class="n">username</span><span class="p">,</span> <span class="n">password …</span></code></pre></div><p>这两天为用bottle+mongodb写的一个项目加上登录功能,无奈怎么都获取不到保存的cookie,文档给出让我们这样操作cookie的代码片段:</p>
<div class="highlight"><pre><span></span><code><span class="nd">@route</span><span class="p">(</span><span class="err">’</span><span class="o">/</span><span class="n">login</span><span class="err">’</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">login</span> <span class="p">():</span>
<span class="n">username</span> <span class="o">=</span> <span class="n">request</span> <span class="o">.</span><span class="n">forms</span> <span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="err">’</span><span class="n">username</span> <span class="err">’</span><span class="p">)</span>
<span class="n">password</span> <span class="o">=</span> <span class="n">request</span> <span class="o">.</span><span class="n">forms</span> <span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="err">’</span><span class="n">password</span> <span class="err">’</span><span class="p">)</span>
<span class="k">if</span> <span class="n">check_user_credentials</span><span class="p">(</span><span class="n">username</span><span class="p">,</span> <span class="n">password</span><span class="p">):</span>
<span class="n">response</span> <span class="o">.</span><span class="n">set_cookie</span><span class="p">(</span><span class="s2">"account"</span><span class="p">,</span> <span class="n">username</span><span class="p">,</span> <span class="n">secret</span><span class="o">=</span> <span class="err">’</span><span class="n">some</span><span class="o">-</span><span class="n">secret</span><span class="o">-</span><span class="n">key</span><span class="err">’</span><span class="p">)</span>
<span class="k">return</span> <span class="s2">"Welcome </span><span class="si">%s</span><span class="s2">!You are now logged in."</span> <span class="o">%</span> <span class="n">username</span>
<span class="k">else</span> <span class="p">:</span>
<span class="k">return</span> <span class="s2">"Login failed."</span>
<span class="nd">@route</span><span class="p">(</span><span class="err">’</span><span class="o">/</span><span class="n">restricted</span><span class="err">’</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">restricted_area</span> <span class="p">():</span>
<span class="n">username</span> <span class="o">=</span> <span class="n">request</span> <span class="o">.</span><span class="n">get_cookie</span><span class="p">(</span><span class="s2">"account"</span><span class="p">,</span> <span class="n">secret</span><span class="o">=</span> <span class="err">’</span><span class="n">some</span><span class="o">-</span><span class="n">secret</span><span class="o">-</span><span class="n">key</span><span class="err">’</span><span class="p">)</span>
<span class="k">if</span> <span class="n">username</span><span class="p">:</span>
<span class="k">return</span> <span class="s2">"Hello </span><span class="si">%s</span><span class="s2">.Welcome back."</span> <span class="o">%</span> <span class="n">username</span>
</code></pre></div>
<p>虽然文档上没有但是还有一种操作cookie的方式:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">request</span><span class="p">,</span> <span class="n">response</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/login'</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s2">"POST"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">login</span><span class="p">():</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">POST</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
<span class="n">passwd</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">POST</span><span class="p">[</span><span class="s1">'passwd'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">check_user_right</span><span class="p">(</span><span class="n">user</span><span class="p">,</span><span class="n">passwd</span><span class="p">):</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'account'</span><span class="p">]</span> <span class="o">=</span> <span class="n">user</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">pass</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/admin'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">admin</span><span class="p">():</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">user</span><span class="p">:</span>
<span class="k">pass</span>
</code></pre></div>
<p>但是无论我用哪种方式操作我都无法获取cookie,为什么呢.百思不得其解.但是我的一个处理文章点击率的提醒了我,代码如下:</p>
<div class="highlight"><pre><span></span><code><span class="nd">@route</span><span class="p">(</span><span class="s1">'/archrives/:aid#\d+#'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">article_show</span><span class="p">(</span><span class="n">aid</span><span class="p">):</span>
<span class="n">db</span> <span class="o">=</span> <span class="n">dbconn</span><span class="o">.</span><span class="n">ConnDB</span><span class="p">()</span>
<span class="n">artid</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">aid</span><span class="p">)</span>
<span class="c1"># 获取客户端ip</span>
<span class="n">remoteip</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'REMOTE_ADDR'</span><span class="p">)</span>
<span class="n">artcookie</span> <span class="o">=</span> <span class="n">remoteip</span><span class="o">+</span><span class="s1">'ip'</span><span class="o">+</span><span class="n">aid</span>
<span class="nb">print</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
<span class="c1"># 判断cookie是否存在</span>
<span class="k">if</span> <span class="n">artcookie</span> <span class="ow">in</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
<span class="c1"># 存在则更新有效时间</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="n">artcookie</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="n">artcookie</span><span class="p">][</span><span class="s1">'max-age'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">500</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># 不存在则更新文章查看次数</span>
<span class="n">db</span><span class="o">.</span><span class="n">posts</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">"id"</span><span class="p">:</span><span class="n">artid</span><span class="p">},</span> <span class="p">{</span><span class="s2">"$inc"</span><span class="p">:{</span><span class="s2">"views"</span><span class="p">:</span><span class="mi">1</span><span class="p">}})</span>
<span class="c1"># 并设置cookie</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="n">artcookie</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="n">artcookie</span><span class="p">][</span><span class="s1">'max-age'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">500</span>
<span class="n">TEMPLATE</span><span class="p">[</span><span class="s1">'posts'</span><span class="p">]</span> <span class="o">=</span> <span class="n">getArtList</span><span class="p">({</span><span class="s2">"id"</span><span class="p">:</span><span class="n">artid</span><span class="p">})</span>
<span class="n">TEMPLATE</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">setTempVar</span><span class="p">())</span>
<span class="k">return</span> <span class="n">template</span><span class="p">(</span><span class="s1">'article.html'</span><span class="p">,</span> <span class="n">TEMPLATE</span><span class="p">)</span>
</code></pre></div>
<p>这里是可以正常获取到cookie的,而且代码没有任何区别.唯一的区别就是用户认证是跳转了页面.所以我help了一下:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">response</span>
<span class="n">help</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">set_cookie</span><span class="p">)</span>
</code></pre></div>
<p>help的结果其中有两个参数一个是path,和domain:</p>
<div class="highlight"><pre><span></span><code> :param domain: the domain that is allowed to read the cookie.
(default: current domain)
:param path: limits the cookie to a given path (default: current path)
</code></pre></div>
<p>明显bottle的cookie默认只在当前路径下能读取的到,所以要别的页面读取到cookie我们的代码须改成如下:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">request</span><span class="p">,</span> <span class="n">response</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/login'</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s2">"POST"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">login</span><span class="p">():</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">POST</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
<span class="n">passwd</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">POST</span><span class="p">[</span><span class="s1">'passwd'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">check_user_right</span><span class="p">(</span><span class="n">user</span><span class="p">,</span><span class="n">passwd</span><span class="p">):</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'account'</span><span class="p">]</span> <span class="o">=</span> <span class="n">user</span>
<span class="n">response</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'account'</span><span class="p">][</span><span class="s1">'path'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'/admin'</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">pass</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/admin'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">admin</span><span class="p">():</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">COOKIES</span><span class="p">[</span><span class="s1">'user'</span><span class="p">]</span>
</code></pre></div>
<p>这样我们就能在别的路径下访问我们设定的cookie.</p>使用beaker让bottle支持session2012-05-26T11:38:00+08:002012-05-26T11:38:00+08:00coldtag:www.linuxzen.com,2012-05-26:/shi-yong-beakerrang-bottlezhi-chi-session.html<p>bottle是一个小型web框架,很小只有一个文件,但功能确很强大,学起来也简单,简单和小巧的同时也有很多不足,某些功能支持还不是很完善,比如 …</p><p>bottle是一个小型web框架,很小只有一个文件,但功能确很强大,学起来也简单,简单和小巧的同时也有很多不足,某些功能支持还不是很完善,比如session.但是也有它自身的好处,我们可以自己或使用别的模块来扩展它,不像django,很强大,但是想要进一步扩展的时候确无从下手.我们可以把非常简单而强大的bottle自己动手将它变得更加强大和完善.</p>
<p>bottle小巧支持cookie但是不支持session.为了安全起见我们有时候希望使用的session.我们可以使用中间件beaker来扩展bottle,使我们的bottle应用支持session.废话不多说.首先beaker不是内置模块,我们首先来安装它.当然你可以网上下包手动安装,我们使用最简单的:</p>
<div class="highlight"><pre><span></span><code>easy_install<span class="w"> </span>beaker
</code></pre></div>
<p>没有easy_install这个命令?google吧,装了之后还是没有,如过时win的话检查环境变量,将Python安装目录下的Scripts目录添加到环境变量.</p>
<p>安装好后我们如何使用它,下面一段带面是使用的:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">route</span><span class="p">,</span> <span class="n">default_app</span><span class="p">,</span> <span class="n">run</span><span class="p">,</span> <span class="n">request</span>
<span class="kn">from</span> <span class="nn">beaker.middleware</span> <span class="kn">import</span> <span class="n">SessionMiddleware</span>
<span class="n">session_opts</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'session.type'</span><span class="p">:</span><span class="s1">'file'</span><span class="p">,</span>
<span class="s1">'session.cookei_expires'</span><span class="p">:</span><span class="mi">300</span><span class="p">,</span>
<span class="s1">'session.data_dir'</span><span class="p">:</span><span class="s1">'./sessions'</span><span class="p">,</span>
<span class="s1">'sessioni.auto'</span><span class="p">:</span><span class="kc">True</span>
<span class="p">}</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/test'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'beaker.session'</span><span class="p">)</span>
<span class="n">s</span><span class="p">[</span><span class="s1">'test'</span><span class="p">]</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'test'</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">s</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="k">return</span> <span class="s1">'Test conter: </span><span class="si">%d</span><span class="s1">'</span> <span class="o">%</span> <span class="n">s</span><span class="p">[</span><span class="s1">'test'</span><span class="p">]</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">default_app</span><span class="p">()</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">SessionMiddleware</span><span class="p">(</span><span class="n">app</span><span class="p">,</span> <span class="n">session_opts</span><span class="p">)</span>
<span class="n">run</span><span class="p">(</span><span class="n">app</span><span class="o">=</span><span class="n">app</span><span class="p">)</span>
</code></pre></div>
<p>运行这段代码,会提示:</p>
<div class="highlight"><pre><span></span><code>Bottle server starting up (using WSGIRefServer())...
Listening on http://127.0.0.1:8080/
Hit Ctrl-C to quit.
</code></pre></div>
<p>现在打开浏览器访问<code>http://127.0.0.1:8080/test</code></p>
<p>不断刷新就会发现数值不断在增大.说明我们的session已经正常工作了</p>使用Linux shell实时检测文件变更2012-05-18T11:16:00+08:002012-05-18T11:16:00+08:00coldtag:www.linuxzen.com,2012-05-18:/shi-yong-linux-shellshi-shi-jian-ce-wen-jian-bian-geng.html<p>使用python做web开发,现在流行使用uwsgi调用python程序,但是使用uwsgi一段时间发现有一个弊端,就是每次更改源代码后必须重启uwsgi才能生效,包括更改模板文件也是,我是个懒人,再经过一段 …</p><p>使用python做web开发,现在流行使用uwsgi调用python程序,但是使用uwsgi一段时间发现有一个弊端,就是每次更改源代码后必须重启uwsgi才能生效,包括更改模板文件也是,我是个懒人,再经过一段时间反复的更改-重启后我终于忍受不了,决定写一个脚本来定时程序目录的文件改动,并及时自动重启uwsgi,来解放我的双手可以不用理会这些琐碎的重启工作. 用了点时间来编写了一个脚本用来判断是否更改,然后判断是否需要重启uwsgi.</p>
<p>下面放出脚本内容:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1"># Author : cold</span>
<span class="c1"># Homepage : http://www.linuxzen.com</span>
<span class="c1"># Filename : checkchange.sh</span>
<span class="c1"># Useage : sh checkchange.sh [dir]</span>
checkisdir<span class="o">()</span>
<span class="w"> </span><span class="c1"># Have one argument</span>
<span class="w"> </span><span class="c1"># The argument is a directory</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>ls<span class="w"> </span><span class="nv">$1</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'s/ /\n/g'</span><span class="sb">`</span>
<span class="w"> </span><span class="k">do</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span>-d<span class="w"> </span><span class="nv">$1</span>/<span class="nv">$i</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"bin"</span><span class="w"> </span>-o<span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"lib"</span><span class="w"> </span>-o<span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"include"</span><span class="w"> </span><span class="o">]</span><span class="w"> </span><span class="c1"># 不想检测的目录(这里是使用virtualenv生成的环境文件)</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="k">continue</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span><span class="nv">dir</span><span class="o">=</span><span class="s2">"</span><span class="nv">$1</span><span class="s2">/</span><span class="nv">$i</span><span class="s2">"</span>
<span class="w"> </span>checkisdir<span class="w"> </span><span class="nv">$dir</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="nv">files</span><span class="o">=</span><span class="nv">$files</span><span class="s1">'\n'</span><span class="nv">$1</span><span class="s1">'/'</span><span class="nv">$i</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span><span class="k">done</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="nv">$files</span>
<span class="o">}</span>
<span class="k">while</span><span class="w"> </span><span class="nb">true</span>
<span class="k">do</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span>-e<span class="w"> </span>/tmp/stat.tmp<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>checkisdir<span class="w"> </span><span class="nv">$1</span><span class="sb">`</span>
<span class="w"> </span><span class="k">do</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span>-e<span class="w"> </span>/tmp/patch.tmp<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>stat<span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>Change<span class="w"> </span>><span class="w"> </span>/tmp/nstat.tmp
<span class="w"> </span>rm<span class="w"> </span>-f<span class="w"> </span>/tmp/patch.tmp
<span class="w"> </span><span class="k">continue</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span>stat<span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>Change<span class="w"> </span>>><span class="w"> </span>/tmp/nstat.tmp
<span class="w"> </span><span class="k">done</span>
<span class="w"> </span>diff<span class="w"> </span>/tmp/stat.tmp<span class="w"> </span>/tmp/nstat.tmp<span class="w"> </span>><span class="w"> </span>/tmp/patch.tmp
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>sleep<span class="w"> </span><span class="m">10</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span>/etc/init.d/uwsgi.py<span class="w"> </span>restart<span class="w"> </span><span class="c1"># 将此处更改为想要做的操作</span>
<span class="w"> </span>patch<span class="w"> </span>/tmp/stat.tmp<span class="w"> </span>/tmp/patch.tmp
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>checkisdir<span class="w"> </span><span class="nv">$1</span><span class="sb">`</span>
<span class="w"> </span><span class="k">do</span>
<span class="w"> </span>stat<span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>Change<span class="w"> </span>>><span class="w"> </span>/tmp/stat.tmp
<span class="w"> </span><span class="k">done</span>
<span class="w"> </span><span class="k">continue</span>
<span class="w"> </span><span class="k">fi</span>
<span class="k">done</span>
</code></pre></div>
<p>这里主要测试变更后重启uwsgi,使用方法:我的bottle程序在/code/python下:</p>
<div class="highlight"><pre><span></span><code>sh<span class="w"> </span>checkchange.sh<span class="w"> </span>/code/python<span class="w"> </span><span class="p">&</span>
</code></pre></div>
<p>如果使用svn可以参考下面代码:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1"># Author : cold</span>
<span class="c1"># Homepage : http://www.linuxzen.com</span>
<span class="c1"># Filename : checkupdate.sh</span>
<span class="c1"># Describle : To Check update of svn</span>
<span class="k">while</span><span class="w"> </span><span class="nb">true</span>
<span class="k">do</span>
<span class="w"> </span><span class="nb">cd</span><span class="w"> </span>/code/python
<span class="w"> </span>svn<span class="w"> </span>up<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>At<span class="w"> </span>><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">2</span>><span class="p">&</span><span class="m">1</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>sleep<span class="w"> </span><span class="m">30</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span>svn<span class="w"> </span>up<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>Updated<span class="w"> </span>><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">2</span>><span class="p">&</span><span class="m">1</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>/etc/init.d/uwsgi.py<span class="w"> </span>restart
<span class="w"> </span><span class="k">fi</span>
<span class="k">done</span>
</code></pre></div>Gentoo下搭建python web环境(nginx+bottle+virtualenv+uwsgi)2012-05-16T11:05:00+08:002012-05-16T11:05:00+08:00coldtag:www.linuxzen.com,2012-05-16:/gentooxia-da-jian-python-webhuan-jing-nginxbottlevirtualenvuwsgi.html<p>最近根据Gentoo官方文档整了一台Gentoo的虚拟机,感觉还是不错的,决定放弃CentOS投奔Gentoo,这几天研究NoSQL mongodb和python的bottle框架,web.py效率不是很好,而且是类级,bottle使用装饰器(虽然对她还是懵懂阶段,但是貌似很强大).感 …</p><p>最近根据Gentoo官方文档整了一台Gentoo的虚拟机,感觉还是不错的,决定放弃CentOS投奔Gentoo,这几天研究NoSQL mongodb和python的bottle框架,web.py效率不是很好,而且是类级,bottle使用装饰器(虽然对她还是懵懂阶段,但是貌似很强大).感觉bottle更加强大和接近python,没有封装太多东西,django只能依照她的思想来做自己的事,最后还是选择了bottle来进入python的web世界,web服务器同样选择nginx.virtualenv可以让一个应用有一个相对独立的环境,特别用于多解释器环境或者经常变更的环境.uwsgi是web和python的中间件(可以这么解释吧).</p>
<h2 id="_1">环境:</h2>
<p>系统:Gentoo
ip:192.168.3.1</p>
<p>好吧废话不多说,Gentoo安装过程这里不再详述,官方文档很详细,下面记录安装配置过程.
首先Gentoo没有默认安装vim,先安装vim:</p>
<div class="highlight"><pre><span></span><code><span class="n">emerge</span> <span class="n">vim</span>
</code></pre></div>
<h3 id="python">安装配置Python</h3>
<p>最新版的Gentoo安装完毕后默认使用python3.2,而我惯用Python2.7.先首先安装python2.7.
Gentoo使用emerge包管理,安装Python2.7:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/portage/dev-lang/python/
emerge<span class="w"> </span>python-2.7.2-r3.ebuild
</code></pre></div>
<p>等待安装完成后使系统默认使用python2.7,先查看默认项</p>
<div class="highlight"><pre><span></span><code>eselect python list
Available Python interpreters:
[1] python2.7
[2] python3.2 *
</code></pre></div>
<p>默认使用3.2,选择python2.7</p>
<div class="highlight"><pre><span></span><code>eselect<span class="w"> </span>python<span class="w"> </span><span class="nb">set</span><span class="w"> </span><span class="m">1</span>
</code></pre></div>
<p>选择过后别忘记执行:</p>
<div class="highlight"><pre><span></span><code>python-updater
</code></pre></div>
<p>好了我已经将python换成了常用的python2.7,下面来安装easy_install:</p>
<div class="highlight"><pre><span></span><code>emerge<span class="w"> </span>setuptools
</code></pre></div>
<p>安装完后使用easy_install安装bottle框架和mongodb驱动:</p>
<div class="highlight"><pre><span></span><code>easy_install<span class="w"> </span>bottle
easy_install<span class="w"> </span>pymogon
</code></pre></div>
<p>安装uwsgi/nginx/virtualenv:</p>
<div class="highlight"><pre><span></span><code>emerge<span class="w"> </span>nginx
emerge<span class="w"> </span>uwsgi
emerge<span class="w"> </span>virtualenv
</code></pre></div>
<h3 id="virtualenv">使用virtualenv创建应用</h3>
<p>我们使用virtualenv创建一个应用:</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>/code/python
virtualenv<span class="w"> </span>/code/python
<span class="nb">source</span><span class="w"> </span>bin/activate
</code></pre></div>
<p>然后查看应用目录下会多出几个目录.</p>
<h3 id="uwsgi">配置uWSGI</h3>
<p>首先创建一个uwsgi配置文件,并编辑它:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>/etc/conf.d/uwsgi<span class="w"> </span>/etc/conf.d/uwsgi.py
vi<span class="w"> </span>/etc/conf.d/uwsgi.py
</code></pre></div>
<p>改成内如如下:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Distributed under the terms of the GNU General Public License v2</span>
<span class="c1"># $Header: /var/cvsroot/gentoo-x86/www-servers/uwsgi/files/uwsgi.confd,v 1.1 2011/05/31 19:49:07 maksbotan Exp $</span>
<span class="c1"># DO NOT MODIFY THIS FILE DIRECTLY! CREATE A COPY AND MODIFY THAT INSTEAD!</span>
<span class="c1"># Path (or name) of UNIX/TCP socket to bind to</span>
<span class="c1">#</span>
<span class="n">UWSGI_SOCKET</span><span class="o">=/</span><span class="k">var</span><span class="o">/</span><span class="n">tmp</span><span class="o">/</span><span class="n">bottle</span><span class="o">.</span><span class="n">sock</span><span class="w"> </span><span class="c1"># 使用unix socket</span>
<span class="c1"># Enable threads?</span>
<span class="c1">#</span>
<span class="n">UWSGI_THREADS</span><span class="o">=</span><span class="mi">1</span>
<span class="c1"># The path to your uWSGI application.</span>
<span class="c1">#</span>
<span class="c1">#UWSGI_PROGRAM=</span>
<span class="c1"># The path to your uWSGI xml config file.</span>
<span class="c1">#</span>
<span class="n">UWSGI_XML_CONFIG</span><span class="o">=/</span><span class="n">code</span><span class="o">/</span><span class="n">bottle</span><span class="o">.</span><span class="n">xml</span><span class="w"> </span><span class="c1"># 使用xml配置文件</span>
<span class="c1"># The number of child processes to spawn. The default is 1.</span>
<span class="c1">#</span>
<span class="n">UWSGI_CHILDREN</span><span class="o">=</span><span class="mi">1</span>
<span class="c1"># The log file path. If empty logging is disabled</span>
<span class="c1">#</span>
<span class="n">UWSGI_LOG_FILE</span><span class="o">=/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">uwsgi</span><span class="o">.</span><span class="n">log</span><span class="w"> </span><span class="c1"># 定义日志</span>
<span class="c1"># If you want to run your application inside a chroot then specify the</span>
<span class="c1"># directory here. Leave this blank otherwise.</span>
<span class="c1">#</span>
<span class="c1">#UWSGI_CHROOT=/code/python</span>
<span class="c1"># If you want to run your application from a specific directiory specify</span>
<span class="c1"># it here. Leave this blank otherwise.</span>
<span class="c1">#</span>
<span class="c1"># UWSGI_DIR=</span>
<span class="c1"># The user and group to run your application as. If you do not specify these,</span>
<span class="c1"># the application will be run as root:root.</span>
<span class="c1">#</span>
<span class="n">UWSGI_USER</span><span class="o">=</span>
<span class="c1"># Additional options you might want to pass to uWSGI</span>
<span class="c1">#</span>
<span class="c1">#UWSGI_EXTRA_OPTIONS=</span>
</code></pre></div>
<p>然后创建bottle.xml配置文件:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/code/bottle.xml
</code></pre></div>
<p>内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="nt"><uwsgi></span>
<span class="w"> </span><span class="nt"><socket></span>/var/tmp/bottle.sock<span class="nt"></socket></span>
<span class="w"> </span><span class="nt"><home></span>/code/python<span class="nt"></home></span>
<span class="w"> </span><span class="nt"><chdir></span>/code/python<span class="nt"></chdir></span>
<span class="w"> </span><span class="nt"><python-path></span>/code/python<span class="nt"></python-path></span>
<span class="w"> </span><span class="nt"><module></span>index<span class="nt"></module></span>
<span class="w"> </span><span class="nt"><master/></span>
<span class="w"> </span><span class="nt"><memory/></span>
<span class="w"> </span><span class="nt"><logto></span>/var/log/bottle.log<span class="nt"></logto></span>
<span class="w"> </span><span class="nt"><daemonize></span>/var/log/uwsgi.log<span class="nt"></daemonize></span>
<span class="nt"></uwsgi></span>
</code></pre></div>
<p>其中<module></module>定义的就是你的web程序,也就是/code/python下要有一个index.py.</p>
<p>配置完毕后创建启动脚本:</p>
<div class="highlight"><pre><span></span><code>ln -sf /etc/init.d/uwsgi /etc/init.d/uwsgi.py
</code></pre></div>
<p>并加入开机启动:</p>
<div class="highlight"><pre><span></span><code>rc-update add uwsgi.py default
</code></pre></div>
<h3 id="nginx">配置nginx</h3>
<p>编辑配置文件:</p>
<div class="highlight"><pre><span></span><code>vi /etc/nginx/nginx.conf
</code></pre></div>
<p>修改内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="n">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">listen</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.1</span><span class="p">:</span><span class="mi">80</span><span class="p">;</span>
<span class="w"> </span><span class="n">server_name</span><span class="w"> </span><span class="n">linuxzen</span><span class="o">.</span><span class="n">com</span><span class="p">;</span>
<span class="w"> </span><span class="n">access_log</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">nginx</span><span class="o">/</span><span class="n">localhost</span><span class="o">.</span><span class="n">access_log</span><span class="w"> </span><span class="n">main</span><span class="p">;</span>
<span class="w"> </span><span class="n">error_log</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">nginx</span><span class="o">/</span><span class="n">localhost</span><span class="o">.</span><span class="n">error_log</span><span class="w"> </span><span class="n">info</span><span class="p">;</span>
<span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="o">/</span><span class="n">code</span><span class="o">/</span><span class="n">python</span><span class="p">;</span>
<span class="w"> </span><span class="n">location</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="p">{</span><span class="w"> </span>
<span class="w"> </span><span class="n">include</span><span class="w"> </span><span class="n">uwsgi_params</span><span class="p">;</span>
<span class="w"> </span><span class="n">uwsgi_param</span><span class="w"> </span><span class="n">UWSGI_PYTHOME</span><span class="w"> </span><span class="o">/</span><span class="n">code</span><span class="o">/</span><span class="n">python</span><span class="p">;</span>
<span class="w"> </span><span class="n">uwsgi_param</span><span class="w"> </span><span class="n">UWSGI_CHDIR</span><span class="w"> </span><span class="o">/</span><span class="n">code</span><span class="o">/</span><span class="n">python</span><span class="p">;</span><span class="w"> </span>
<span class="w"> </span><span class="n">uwsgi_param</span><span class="w"> </span><span class="n">UWSGI_SCRIPT</span><span class="w"> </span><span class="n">index</span><span class="p">;</span>
<span class="w"> </span><span class="n">uwsgi_pass</span><span class="w"> </span><span class="n">unix</span><span class="p">:</span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">tmp</span><span class="o">/</span><span class="n">bottle</span><span class="o">.</span><span class="n">sock</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span><span class="w"> </span>
<span class="w"> </span><span class="p">}</span>
</code></pre></div>
<p>同样将nginx作为开机启动:</p>
<div class="highlight"><pre><span></span><code>rc-update add nginx default
</code></pre></div>
<h2 id="_2">测试</h2>
<p>首先创建一个bottle程序供测试:</p>
<div class="highlight"><pre><span></span><code><span class="n">vi</span> <span class="o">/</span><span class="n">code</span><span class="o">/</span><span class="n">python</span><span class="o">/</span><span class="n">index</span><span class="o">.</span><span class="n">py</span>
<span class="n">参考内容如下</span><span class="p">:</span>
<span class="c1">#!/usr/bin/env python</span>
<span class="c1">#-*- coding: utf8 -*-</span>
<span class="sd">'''</span>
<span class="sd"># Author : cold night</span>
<span class="sd"># Filename : index.py </span>
<span class="sd">'''</span>
<span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">run</span><span class="p">,</span> <span class="n">route</span><span class="p">,</span> <span class="n">default_app</span>
<span class="nd">@route</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">index</span><span class="p">():</span>
<span class="k">return</span> <span class="s2">"Hello world!"</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s2">"localhost"</span><span class="p">,</span> <span class="n">port</span><span class="o">=</span><span class="mi">8888</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">application</span> <span class="o">=</span> <span class="n">default_app</span><span class="p">()</span>
</code></pre></div>
<p>然后启动nginx和uwsgi:</p>
<div class="highlight"><pre><span></span><code>/etc/init.d/nginx start
/etc/init.d/uwsgi.py start
</code></pre></div>
<p>然后通过浏览器访问即可看到<code>Hello world!</code></p>iptables:restorecon: command not found无法保存策略解决2012-04-30T10:43:00+08:002012-04-30T10:43:00+08:00coldtag:www.linuxzen.com,2012-04-30:/iptablesrestorecon-command-not-foundwu-fa-bao-cun-ce-lue-jie-jue.html<p>今天更新了几条iptables,但是运行命令<code>service iptables save</code>保存时确提示<code>iptables: Saving firewall rules to /etc/sysconfig/iptables: /etc/init.d/iptables: line 268: restorecon: command not found</code></p>
<p>好吧,很明显的错误,找 …</p><p>今天更新了几条iptables,但是运行命令<code>service iptables save</code>保存时确提示<code>iptables: Saving firewall rules to /etc/sysconfig/iptables: /etc/init.d/iptables: line 268: restorecon: command not found</code></p>
<p>好吧,很明显的错误,找不到restorecon命令. 查找一下:</p>
<div class="highlight"><pre><span></span><code>which restorecon
whereis restorecon
</code></pre></div>
<p>没结果,继续手动查看:</p>
<div class="highlight"><pre><span></span><code><span class="n">ls</span><span class="w"> </span><span class="o">-</span><span class="n">l</span><span class="w"> </span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="w"> </span><span class="err">¦</span><span class="w"> </span><span class="n">grep</span><span class="w"> </span><span class="n">restore</span>
<span class="n">lrwxrwxrwx</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="mi">14</span><span class="w"> </span><span class="n">Apr</span><span class="w"> </span><span class="mi">19</span><span class="w"> </span><span class="mi">09</span><span class="o">:</span><span class="mi">58</span><span class="w"> </span><span class="n">iptables</span><span class="o">-</span><span class="n">restore</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">iptables</span><span class="o">-</span><span class="n">multi</span>
</code></pre></div>
<p>有一条但是不是我们想要的,猜测大概是包没装
先yum搜索一下吧:</p>
<div class="highlight"><pre><span></span><code>yum search restorecon
</code></pre></div>
<p>好吧依然没有结果.google一下说是缺少policycoreutils这个包.安装这个包:</p>
<div class="highlight"><pre><span></span><code>yum -y install policycoreutils
</code></pre></div>
<p>然后查看:</p>
<div class="highlight"><pre><span></span><code><span class="n">ls</span><span class="w"> </span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="w"> </span><span class="o">-</span><span class="n">l</span><span class="w"> </span><span class="err">¦</span><span class="n">grep</span><span class="w"> </span><span class="n">restore</span>
<span class="n">lrwxrwxrwx</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="mi">14</span><span class="w"> </span><span class="n">Apr</span><span class="w"> </span><span class="mi">19</span><span class="w"> </span><span class="mi">09</span><span class="o">:</span><span class="mi">58</span><span class="w"> </span><span class="n">iptables</span><span class="o">-</span><span class="n">restore</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">iptables</span><span class="o">-</span><span class="n">multi</span>
<span class="n">lrwxrwxrwx</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="mi">8</span><span class="w"> </span><span class="n">Apr</span><span class="w"> </span><span class="mi">30</span><span class="w"> </span><span class="mi">10</span><span class="o">:</span><span class="mi">31</span><span class="w"> </span><span class="n">restorecon</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">setfiles</span>
</code></pre></div>
<p>然后运行命令保存iptables策略:</p>
<div class="highlight"><pre><span></span><code>service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
</code></pre></div>
<p>只是一篇笔记,记录问题方便以后遇到问题后查找.有点乱.没多少内容.</p>用Python写的终端下的翻译工具2012-04-23T23:23:00+08:002012-04-23T23:23:00+08:00coldtag:www.linuxzen.com,2012-04-23:/yong-pythonxie-de-zhong-duan-xia-de-fan-yi-gong-ju.html<p>为什么写这个程序,为什么不给这个程序配备gui?原因很简单,因为我是一个命令行控,Linux习惯了不习惯了鼠标,总觉得点着不如敲命令快,各位在 …</p><p>为什么写这个程序,为什么不给这个程序配备gui?原因很简单,因为我是一个命令行控,Linux习惯了不习惯了鼠标,总觉得点着不如敲命令快,各位在看这篇文章就说明和本人有相同的爱好.这个用python写的翻译工具是通过google来实现的,由于google返回的数据不是很规范(或者说我没有找到规律),现在前三项能正常显示(源词,翻译结果,和汉语拼音).下面的词性和其他释义可能不同,见谅,望大神可以指点下小弟和帮小弟完善,这里赶紧不尽.</p>
<p>好了不费话了,下面放代码:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># -*-coding:utf8 -*-</span>
<span class="sd">'''</span>
<span class="sd">#=============================================================================</span>
<span class="sd"># FileName: translate.py</span>
<span class="sd"># Desc: To translate with zh to en or en2zh</span>
<span class="sd"># Author: cold</span>
<span class="sd"># Email: wh_linux@126.com</span>
<span class="sd"># HomePage: http://www.linuxzen.com</span>
<span class="sd"># Version: 0.0.1</span>
<span class="sd"># LastChange: 2012-04-23 23:04:08</span>
<span class="sd"># History:</span>
<span class="sd">#=============================================================================</span>
<span class="sd">'''</span>
<span class="kn">import</span> <span class="nn">urllib</span>
<span class="kn">import</span> <span class="nn">urllib2</span>
<span class="kn">from</span> <span class="nn">sys</span> <span class="kn">import</span> <span class="n">argv</span><span class="p">,</span><span class="n">exit</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="c1"># 显示帮助信息</span>
<span class="k">def</span> <span class="nf">helpinfo</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'''</span>
<span class="s1">Usage: pytran {zh2en¦en2zh} content</span>
<span class="s1">'''</span>
<span class="c1"># 格式化输出</span>
<span class="k">def</span> <span class="nf">formatresult</span><span class="p">(</span><span class="n">result</span><span class="p">,</span><span class="n">srclang</span><span class="p">):</span>
<span class="n">resu</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'[['</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">srclang</span><span class="o">==</span><span class="s1">'en2zh'</span> <span class="ow">or</span> <span class="n">srclang</span> <span class="o">==</span> <span class="s1">'zh2en'</span><span class="p">):</span>
<span class="n">firstre</span> <span class="o">=</span> <span class="n">resu</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'['</span><span class="p">,</span><span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">']'</span><span class="p">,</span><span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'"'</span><span class="p">)</span>
<span class="nb">print</span> <span class="s1">'源词:'</span><span class="p">,</span><span class="n">firstre</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span>
<span class="nb">print</span> <span class="s1">'结果:'</span><span class="p">,</span><span class="n">firstre</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">if</span> <span class="p">(</span><span class="n">srclang</span><span class="o">==</span><span class="s1">'zh2en'</span><span class="p">):</span>
<span class="n">piny</span> <span class="o">=</span> <span class="n">firstre</span><span class="p">[</span><span class="mi">7</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">piny</span> <span class="o">=</span> <span class="n">firstre</span><span class="p">[</span><span class="mi">5</span><span class="p">]</span>
<span class="nb">print</span> <span class="s1">'拼音:'</span><span class="p">,</span><span class="n">piny</span>
<span class="k">if</span><span class="p">(</span><span class="n">srclang</span><span class="o">==</span><span class="s1">'zh2en'</span><span class="p">):</span>
<span class="n">secresu</span><span class="o">=</span><span class="n">resu</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'"'</span><span class="p">,</span><span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'['</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">secresu</span> <span class="o">=</span> <span class="n">resu</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'"'</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'['</span><span class="p">)</span>
<span class="nb">print</span> <span class="s1">'词性:'</span><span class="p">,</span><span class="n">secresu</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">','</span><span class="p">,</span><span class="s1">''</span><span class="p">)</span>
<span class="nb">print</span> <span class="s1">'其他释义:'</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="s1">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">secresu</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">']'</span><span class="p">))</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">','</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">i</span>
<span class="c1"># 获取命令行参数</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">srclang</span> <span class="o">=</span> <span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">helpinfo</span><span class="p">()</span>
<span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">cont</span> <span class="o">=</span> <span class="n">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">helpinfo</span><span class="p">()</span>
<span class="n">exit</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="c1"># 判断翻译目标语言用来确定传送参数</span>
<span class="k">if</span><span class="p">(</span><span class="n">srclang</span> <span class="o">==</span> <span class="s1">'zh2en'</span><span class="p">):</span>
<span class="n">data</span><span class="o">=</span><span class="n">urllib</span><span class="o">.</span><span class="n">urlencode</span><span class="p">({</span><span class="s1">'client'</span><span class="p">:</span><span class="s1">'t'</span><span class="p">,</span> <span class="s1">'text'</span><span class="p">:</span><span class="n">cont</span><span class="p">,</span>
<span class="s1">'hl'</span><span class="p">:</span><span class="s1">'zh-CN'</span><span class="p">,</span><span class="s1">'tl'</span><span class="p">:</span><span class="s1">'en'</span><span class="p">,</span>
<span class="s1">'multires'</span><span class="p">:</span><span class="s1">'1'</span><span class="p">,</span><span class="s1">'prev'</span><span class="p">:</span><span class="s1">'btn'</span><span class="p">,</span>
<span class="s1">'ssel'</span><span class="p">:</span><span class="s1">'0'</span><span class="p">,</span><span class="s1">'sc'</span><span class="p">:</span><span class="s1">'1'</span><span class="p">})</span>
<span class="k">elif</span><span class="p">(</span><span class="n">srclang</span> <span class="o">==</span> <span class="s1">'en2zh'</span><span class="p">):</span>
<span class="n">data</span><span class="o">=</span><span class="n">urllib</span><span class="o">.</span><span class="n">urlencode</span><span class="p">({</span><span class="s1">'client'</span><span class="p">:</span><span class="s1">'t'</span><span class="p">,</span> <span class="s1">'text'</span><span class="p">:</span><span class="n">cont</span><span class="p">,</span>
<span class="s1">'hl'</span><span class="p">:</span><span class="s1">'zh-CN'</span><span class="p">,</span> <span class="s1">'sl'</span><span class="p">:</span><span class="s1">'en'</span><span class="p">,</span><span class="s1">'tl'</span><span class="p">:</span><span class="s1">'zh-CN'</span><span class="p">,</span>
<span class="s1">'multires'</span><span class="p">:</span><span class="s1">'1'</span><span class="p">,</span> <span class="s1">'prev'</span><span class="p">:</span><span class="s1">'btn'</span><span class="p">,</span>
<span class="s1">'ssel'</span><span class="p">:</span><span class="s1">'0'</span><span class="p">,</span><span class="s1">'sc'</span><span class="p">:</span><span class="s1">'1'</span><span class="p">})</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">helpinfo</span><span class="p">()</span>
<span class="c1"># 打开google翻译内容</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://translate.google.cn/translate_a/t'</span>
<span class="n">req</span> <span class="o">=</span><span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">,</span><span class="n">data</span><span class="p">)</span>
<span class="n">req</span><span class="o">.</span><span class="n">add_header</span><span class="p">(</span><span class="s2">"User-Agent"</span><span class="p">,</span> <span class="s2">"Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)"</span><span class="p">)</span>
<span class="n">fd</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">req</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">fd</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="c1"># 格式化输出</span>
<span class="n">formatresult</span><span class="p">(</span><span class="n">result</span><span class="p">,</span> <span class="n">srclang</span><span class="p">)</span>
<span class="n">fd</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</code></pre></div>
<p>为了更方便的使用我们把这个脚本命名位pytranslate,放到/usr/bin下,并赋予执行权限:</p>
<div class="highlight"><pre><span></span><code>chmod +x /usr/bin/pytranslate
</code></pre></div>
<p>然后我们就可以使用它进行翻译了:
翻译英文到中文:</p>
<div class="highlight"><pre><span></span><code>pytranslate en2zh extent
源词: extent
结果: 程度
拼音: Chéngdù
词性: 名词
其他释义:
程度
范围
幅度
规模
度
地步
广度
长度
面
长短
份儿
界
en
翻译中文到英文
pytranslate zh2en 中国
源词: 中国
结果: China
拼音: Zhōngguó
词性: 名词
其他释义:
China
zh-CN
</code></pre></div>
<p>好吧相信聪明的你肯定发现如何使用了这里就不罗嗦了.</p>用python发带附件的邮件用来定时备份mysql数据库2012-04-21T18:26:00+08:002012-04-21T18:26:00+08:00coldtag:www.linuxzen.com,2012-04-21:/yong-pythonfa-dai-fu-jian-de-you-jian-yong-lai-ding-shi-bei-fen-mysqlshu-ju-ku.html<p>最近迁移了wordpress,系统升级为CentOS 6,很奇怪的一个问题,在原来CentOS 5.8下用的很正常的定时备份数据库并通过邮件发送的脚本不能发送附件,其他都正常,邮件内容也是uuencode生成的文件编码,但是就是不产生附件.而且 …</p><p>最近迁移了wordpress,系统升级为CentOS 6,很奇怪的一个问题,在原来CentOS 5.8下用的很正常的定时备份数据库并通过邮件发送的脚本不能发送附件,其他都正常,邮件内容也是uuencode生成的文件编码,但是就是不产生附件.而且找不出原因,望有知道的不吝赐教.</p>
<p>为了解决这一问题,我用Python写了一个mail客户端,可以发送附件,是一个命令行程序.废话不多说.贴代码:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1">#-*- coding: utf8 -*-</span>
<span class="sd">'''</span>
<span class="sd">#=============================================================================</span>
<span class="sd"># FileName: mail.py</span>
<span class="sd"># Desc: To send email</span>
<span class="sd"># Author: cold</span>
<span class="sd"># Email: wh_linux@126.com</span>
<span class="sd"># HomePage: http://www.linuxzen.com</span>
<span class="sd"># Version: 0.0.1</span>
<span class="sd"># LastChange: 2012-04-21 16:37:20</span>
<span class="sd"># History:</span>
<span class="sd">#=============================================================================</span>
<span class="sd">'''</span>
<span class="sd">'''</span>
<span class="sd">用于发送邮件,可以发送附件</span>
<span class="sd">命令行程序</span>
<span class="sd">'''</span>
<span class="kn">import</span> <span class="nn">smtplib</span>
<span class="kn">from</span> <span class="nn">email.mime.text</span> <span class="kn">import</span> <span class="n">MIMEText</span>
<span class="kn">from</span> <span class="nn">email.mime.multipart</span> <span class="kn">import</span> <span class="n">MIMEMultipart</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="c1"># 打印帮助信息</span>
<span class="k">def</span> <span class="nf">helpinfo</span><span class="p">():</span>
<span class="nb">print</span> <span class="s1">'''</span>
<span class="s1"> Useage: pymail -u user@domain -p passwd -h smtp server host -t to who [-a attachment file path] [-n attachment name]</span>
<span class="s1"> Useage: email content use . to end</span>
<span class="s1"> -h specify smtp server host</span>
<span class="s1"> -u which user you login the smtp server,and must with it domain</span>
<span class="s1"> -p the password of the smtp user</span>
<span class="s1"> -t The email recipient,multiple addresses can use ',' split</span>
<span class="s1"> -a Add attachment</span>
<span class="s1"> -n Secify attachment name in the email</span>
<span class="s1"> Author:cold(wh_linux@126.com)</span>
<span class="s1"> Homepge:http://www.linuxzen.com</span>
<span class="s1"> '''</span>
<span class="c1"># 所有选项</span>
<span class="n">options</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'-t'</span><span class="p">,</span> <span class="s1">'-a'</span><span class="p">,</span> <span class="s1">'-n'</span><span class="p">,</span> <span class="s1">'-h'</span><span class="p">,</span> <span class="s1">'-u'</span><span class="p">,</span> <span class="s1">'-p'</span><span class="p">,</span> <span class="s1">'-s'</span><span class="p">]</span>
<span class="c1"># 获取选项长度</span>
<span class="n">argvnum</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span>
<span class="c1"># 检测命令行参数</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">argvnum</span><span class="p">):</span>
<span class="k">if</span> <span class="p">(</span> <span class="n">i</span> <span class="o">%</span><span class="mi">2</span> <span class="o">!=</span> <span class="mi">0</span><span class="p">):</span>
<span class="k">if</span> <span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">options</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'Unknow option '</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="p">,</span> <span class="s1">', Please use -h see help!'</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="c1"># 如果是-h或者没有命令行参数则显示帮助</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'-h'</span> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="n">helpinfo</span><span class="p">()</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">helpinfo</span><span class="p">()</span>
<span class="c1"># 检测-n参数</span>
<span class="k">if</span> <span class="p">(</span><span class="s1">'-n'</span> <span class="ow">in</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">'-a'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'Error:option "-n" must use after -a'</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="c1"># 下面则是获取各个参数内容</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">tmpmailto</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-t'</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">if</span> <span class="s1">','</span> <span class="ow">in</span> <span class="n">tmpmailto</span><span class="p">:</span>
<span class="n">mailto</span> <span class="o">=</span> <span class="n">tmpmailto</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">','</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">mailto</span> <span class="o">=</span> <span class="p">[</span><span class="n">tmpmailto</span><span class="p">,]</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="nb">print</span> <span class="s1">'Error: need Mail Recipient'</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">haveattr</span><span class="o">=</span><span class="kc">True</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">attrpath</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-a'</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">attrname</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-n'</span><span class="p">)</span> <span class="o">+</span><span class="mi">1</span> <span class="p">]</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="n">attrname</span> <span class="o">=</span> <span class="n">attrpath</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">attrname</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">haveattr</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">attrpath</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">mail_host</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-h'</span><span class="p">)</span> <span class="o">+</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="nb">print</span> <span class="s1">'Waring: No specify smtp server use 127.0.0.1'</span>
<span class="n">mail_host</span> <span class="o">=</span> <span class="s1">'127.0.0.1'</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">mail_useremail</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-u'</span><span class="p">)</span> <span class="o">+</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="nb">print</span> <span class="s1">'Waring: No specify user, use root'</span>
<span class="n">mail_useremail</span> <span class="o">=</span> <span class="s1">'root@localhost'</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">mail_sub</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-s'</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">mail_sub</span> <span class="o">=</span> <span class="s1">'No Subject'</span>
<span class="n">mail_user</span> <span class="o">=</span> <span class="n">mail_useremail</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">mail_postfix</span> <span class="o">=</span> <span class="n">mail_useremail</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">mail_pass</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s1">'-p'</span><span class="p">)</span> <span class="o">+</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="n">mail_pass</span> <span class="o">=</span> <span class="s1">''</span>
<span class="c1"># 定义邮件发送函数</span>
<span class="k">def</span> <span class="nf">send_mail</span><span class="p">(</span><span class="n">to_list</span><span class="p">,</span> <span class="n">sub</span><span class="p">,</span> <span class="n">content</span><span class="p">,</span> <span class="n">haveattr</span><span class="p">,</span> <span class="n">attrpath</span><span class="p">,</span> <span class="n">attrname</span><span class="p">):</span>
<span class="n">me</span> <span class="o">=</span> <span class="n">mail_user</span> <span class="o">+</span> <span class="s2">"<"</span> <span class="o">+</span> <span class="n">mail_user</span><span class="o">+</span><span class="s2">"@"</span><span class="o">+</span><span class="n">mail_postfix</span> <span class="o">+</span><span class="s2">">"</span>
<span class="c1"># 判断是否有附件</span>
<span class="k">if</span> <span class="p">(</span><span class="n">haveattr</span><span class="p">):</span>
<span class="k">if</span> <span class="p">(</span><span class="ow">not</span> <span class="n">attrpath</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'Error : no input file of attachments'</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="c1"># 有附件则创建一个带附件的实例</span>
<span class="n">msg</span> <span class="o">=</span> <span class="n">MIMEMultipart</span><span class="p">()</span>
<span class="c1"># 构造附件</span>
<span class="n">att</span> <span class="o">=</span> <span class="n">MIMEText</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="n">attrpath</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">(),</span><span class="s1">'base64'</span><span class="p">,</span> <span class="s1">'utf8'</span><span class="p">)</span>
<span class="n">att</span><span class="p">[</span><span class="s2">"Content-Type"</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'application/octest-stream'</span>
<span class="n">att</span><span class="p">[</span><span class="s2">"Content-Disposition"</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'attachment;filename="'</span><span class="o">+</span> <span class="n">attrname</span> <span class="o">+</span><span class="s1">'"'</span>
<span class="n">msg</span><span class="o">.</span><span class="n">attach</span><span class="p">(</span><span class="n">att</span><span class="p">)</span>
<span class="n">msg</span><span class="o">.</span><span class="n">attach</span><span class="p">(</span><span class="n">MIMEText</span><span class="p">(</span><span class="n">content</span><span class="p">))</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># 无责创建一个文本的实例</span>
<span class="n">msg</span> <span class="o">=</span> <span class="n">MIMEText</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="c1"># 邮件头</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'Subject'</span><span class="p">]</span> <span class="o">=</span> <span class="n">sub</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'From'</span><span class="p">]</span> <span class="o">=</span> <span class="n">me</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'To'</span><span class="p">]</span> <span class="o">=</span> <span class="s2">";"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">to_list</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># 发送邮件</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">smtplib</span><span class="o">.</span><span class="n">SMTP</span><span class="p">()</span>
<span class="n">s</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">mail_host</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">mail_host</span> <span class="o">!=</span> <span class="s1">'127.0.0.1'</span><span class="p">):</span>
<span class="n">s</span><span class="o">.</span><span class="n">login</span><span class="p">(</span><span class="n">mail_user</span><span class="p">,</span> <span class="n">mail_pass</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">sendmail</span><span class="p">(</span><span class="n">me</span><span class="p">,</span> <span class="n">to_list</span><span class="p">,</span> <span class="n">msg</span><span class="o">.</span><span class="n">as_string</span><span class="p">())</span>
<span class="n">s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">return</span> <span class="kc">True</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span>
<span class="nb">print</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">content</span> <span class="o">=</span> <span class="s1">''</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">raw_input</span><span class="p">(</span><span class="s1">''</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">'.'</span><span class="p">:</span>
<span class="k">break</span>
<span class="n">content</span> <span class="o">+=</span> <span class="n">c</span> <span class="o">+</span> <span class="s1">'</span><span class="se">\n</span><span class="s1">'</span>
<span class="k">except</span> <span class="ne">EOFError</span><span class="p">:</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdin</span><span class="p">:</span>
<span class="n">content</span> <span class="o">+=</span> <span class="n">line</span>
<span class="k">if</span> <span class="n">send_mail</span><span class="p">(</span><span class="n">mailto</span><span class="p">,</span> <span class="n">mail_sub</span><span class="p">,</span> <span class="n">content</span><span class="p">,</span> <span class="n">haveattr</span><span class="p">,</span> <span class="n">attrpath</span><span class="p">,</span> <span class="n">attrname</span><span class="p">):</span>
<span class="nb">print</span> <span class="s2">"Success"</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Failed"</span>
</code></pre></div>
<p>将这个脚本保存为pymail放到/usr/bin/下,并赋予其执行权限:</p>
<div class="highlight"><pre><span></span><code>chmod +x /usr/bin/pymail
</code></pre></div>
<p>可以使用 -h指定smtp发件服务器,默认认为指定-h需要认证, 所以就需要smtp服务器支持认证,同时需要-u指定用户名(需加"@域名"),-p指定密码.
如果不指定-h就会使用本地smtp服务器,默认不需要认证,所以本地的smtp服务器就不能支持认证,同时不需指定-u,-p参数</p>
<div class="highlight"><pre><span></span><code>-t 指定收件人多个可用,号分割.
-a 指定附件路径
-n 指定附件名(可省略)
-h 显示帮助信息.
-s 指定邮件主题
</code></pre></div>
<p>执行后会要求输入邮件内容,写完用.结束
也可以用管道下面给出几个实例:</p>
<div class="highlight"><pre><span></span><code><span class="c1">#使用本地smtp服务发送</span>
<span class="nb">echo</span><span class="w"> </span><span class="s1">'linuxzen.com backup'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>pymail<span class="w"> </span>-s<span class="w"> </span><span class="s2">"Linuxzen backup"</span><span class="w"> </span>-t<span class="w"> </span><span class="m">123456</span>@qq.com<span class="w"> </span>-a<span class="w"> </span>/tmp/linuxzen.tar.gz
<span class="c1"># 使用126邮箱发送</span>
<span class="nb">echo</span><span class="w"> </span><span class="s1">'linuxzen.com backup'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>pymail<span class="w"> </span>-u<span class="w"> </span>linuxzen@126.com<span class="w"> </span>-p<span class="w"> </span>linuxzen.com<span class="w"> </span>-h<span class="w"> </span>smtp.126.com<span class="w"> </span>-s<span class="w"> </span><span class="s1">'Linuxzen backup " -t 123456@qq.com -a /tmp/linuxzen.tar.gz </span>
<span class="s1"># 不使用管道发送</span>
<span class="s1">pymail -u linuxzen@126.com -p linuxzen.com -h smtp.126.com -s '</span>hello<span class="w"> </span>world<span class="err">'</span><span class="w"> </span>-t<span class="w"> </span><span class="m">123456</span>@qq.com<span class="w"> </span>-a<span class="w"> </span>/tmp/linuxzen.tar.gz
Hello
this<span class="w"> </span>is<span class="w"> </span>a<span class="w"> </span><span class="nb">test</span><span class="w"> </span>mail
.
</code></pre></div>
<p>下面之前使用的mysql定时备份的脚本:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="nb">export</span><span class="w"> </span><span class="nv">PATH</span><span class="o">=</span>/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
<span class="nv">DATE</span><span class="o">=</span><span class="sb">`</span>date<span class="w"> </span>+%Y%m%d<span class="sb">`</span>
mysqldump<span class="w"> </span>-u<span class="w"> </span>root<span class="w"> </span>blogdata<span class="w"> </span>><span class="w"> </span>/tmp/blogdate.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql
<span class="nb">cd</span><span class="w"> </span>/tmp
tar<span class="w"> </span>-zcf<span class="w"> </span>blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql.tar.gz<span class="w"> </span>blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql
uuencode<span class="w"> </span>blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql.tar.gz<span class="w"> </span>blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql.tar.gz<span class="w"> </span><span class="p">|</span><span class="w"> </span>mail<span class="w"> </span>-s<span class="w"> </span><span class="s1">'MySQL Backup'</span><span class="w"> </span><span class="m">123456</span>@qq.com
</code></pre></div>
<p>没有命令uuencode安装sharutils包即可</p>
<div class="highlight"><pre><span></span><code>yum -y install sharutils
</code></pre></div>
<p>然后使用crontab调用这个脚本定时执行,前面说了 这个脚本在CentOS5.x下正常工作,但是放到CentOS6下就不带附件,所以使用我们自己编写的python脚本脚本内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="nb">export</span><span class="w"> </span><span class="nv">PATH</span><span class="o">=</span>/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
<span class="nv">DATE</span><span class="o">=</span><span class="sb">`</span>date<span class="w"> </span>+%Y%m%d<span class="sb">`</span>
mysqldump<span class="w"> </span>-u<span class="w"> </span>root<span class="w"> </span>blogdata<span class="w"> </span>><span class="w"> </span>/tmp/myblog.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql
<span class="nb">cd</span><span class="w"> </span>/tmp
tar<span class="w"> </span>-zcf<span class="w"> </span>blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql.tar.gz<span class="w"> </span>myblog.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql
<span class="nb">echo</span><span class="w"> </span><span class="s1">'MySQL backup'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>pymail<span class="w"> </span>-u<span class="w"> </span>linuzen@126.com<span class="w"> </span>-p<span class="w"> </span>linuxzen.com<span class="w"> </span>-h<span class="w"> </span>smtp.126.com<span class="w"> </span>-s<span class="w"> </span><span class="s1">'MySQL backup'</span><span class="w"> </span>-a<span class="w"> </span>/tmp/blogdata.<span class="s2">"</span><span class="nv">$DATE</span><span class="s2">"</span>.sql.tar.gz<span class="w"> </span>-t<span class="w"> </span><span class="m">123456</span>@qq.com
</code></pre></div>
<p>我们使用126邮箱来发送 这样就可以把自带的sendmail 停掉:</p>
<div class="highlight"><pre><span></span><code>service sendmail stop
chkonfig --del sendmail
</code></pre></div>CentOS下搭建python web生产环境(nginx+web.py+uwsgi)2012-04-19T17:59:00+08:002012-04-19T17:59:00+08:00coldtag:www.linuxzen.com,2012-04-19:/centosxia-da-jian-python-websheng-chan-huan-jing-nginxwebpyuwsgi.html<p>前面都一篇文章介绍介绍了Ubuntu下web.py的开发环境搭建,这篇文章主要来介绍如何让web.py结合nginx来实现生产环境</p>
<p>首先使用环境介绍:
系统: CentOS 5.5 32位</p>
<p>Python版本:2.7.2</p>
<p>nginx:1.0.13 …</p><p>前面都一篇文章介绍介绍了Ubuntu下web.py的开发环境搭建,这篇文章主要来介绍如何让web.py结合nginx来实现生产环境</p>
<p>首先使用环境介绍:
系统: CentOS 5.5 32位</p>
<p>Python版本:2.7.2</p>
<p>nginx:1.0.13</p>
<p>ip:192.168.3.3/24</p>
<p>由于CentOS默认自带都python(2.4.3)版本较低,所以我们采用手动编译安装python的方式来使用python 2.7.2</p>
<p>首先安装python 2.7.2</p>
<div class="highlight"><pre><span></span><code>cd /usr/src/
mkdir python
cd python
#下载python2.7.2
wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2
tar -jvxf Python-2.7.2.tar.bz2
cd Python-2.7.2
./configure --prefix=/usr/local/python27 --enable-unicode=ucs4
</code></pre></div>
<p>先别急着安装,为什么后面的步骤能顺利进行我们需要我们的python支持zlib模块,</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>Modules/Setup
<span class="c1">#在454行左右找到:#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz</span>
<span class="c1"># 去掉注释</span>
zlib<span class="w"> </span>zlibmodule.c<span class="w"> </span>-I<span class="k">$(</span>prefix<span class="k">)</span>/include<span class="w"> </span>-L<span class="k">$(</span>exec_prefix<span class="k">)</span>/lib<span class="w"> </span>-lz
然后接着编译
make<span class="w"> </span><span class="o">&&</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>安装完成后我们想更方面的使用我们新安装的Python我们做如下更改.</p>
<div class="highlight"><pre><span></span><code>mv<span class="w"> </span>/usr/bin/python<span class="w"> </span>/usr/bin/python24
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/python27/bin/python<span class="w"> </span>/usr/bin/python
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/python27/bin/python2.7<span class="w"> </span>/usr/bin/python2.7
</code></pre></div>
<p>这样改完我们的yum就无法工作了,我们要修改yum来使yum工作:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/usr/bin/yum
</code></pre></div>
<p>将/#!/usr/bin/python改为#!/usr/bin/python2.4即可正常工作(版本可能不一样需查看自己系统自带的版本是什么)</p>
<p>现在我们执行python -V查看应该就是我们刚刚安装的2.7.2版本:</p>
<div class="highlight"><pre><span></span><code>python<span class="w"> </span>-V
Python<span class="w"> </span><span class="m">2</span>.7.2
</code></pre></div>
<p>我们安装了python下面我们就来武装我们的新python,</p>
<p>我们先为新的Python安装python的setuptools,配备easy_install.easy_install用于安装Python第三方扩展包而且只要一个命令即可完成:</p>
<p>下载:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5<span class="o">=</span>fe1f997bc722265116870bc7919059ea
</code></pre></div>
<p>然后直接执行安装:</p>
<div class="highlight"><pre><span></span><code><span class="n">sh</span><span class="w"> </span><span class="n">setuptools</span><span class="o">-</span><span class="mf">0.6</span><span class="n">c11</span><span class="o">-</span><span class="n">py2</span><span class="o">.</span><span class="mf">7.</span><span class="n">egg</span>
</code></pre></div>
<p>安装好之后我们做一个软链接方便我们使用:</p>
<div class="highlight"><pre><span></span><code>ln -s /usr/local/python27/bin/easy_install* /usr/bin/
</code></pre></div>
<p>然后我们来使用easy_install来安装Python第三方扩展</p>
<p>安装本文所需要的web.py</p>
<div class="highlight"><pre><span></span><code>easy_install web.py
</code></pre></div>
<p>然后我们打开Python shell输入</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">web</span>
</code></pre></div>
<p>如果没有报错则说明我们安装成功
安装flup:</p>
<div class="highlight"><pre><span></span><code>easy_install flup
</code></pre></div>
<p>安装Spawn-fcgi :</p>
<div class="highlight"><pre><span></span><code><span class="n">wegt</span><span class="w"> </span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">lighttpd</span><span class="o">.</span><span class="n">net</span><span class="o">/</span><span class="n">download</span><span class="o">/</span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span><span class="o">-</span><span class="mf">1.6</span><span class="o">.</span><span class="mf">3.</span><span class="n">tar</span><span class="o">.</span><span class="n">gz</span>
<span class="n">tar</span><span class="w"> </span><span class="o">-</span><span class="n">zxvf</span><span class="w"> </span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span><span class="o">-</span><span class="mf">1.6</span><span class="o">.</span><span class="mf">3.</span><span class="n">tar</span><span class="o">.</span><span class="n">gz</span>
<span class="n">cd</span><span class="w"> </span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span><span class="o">-</span><span class="mf">1.6</span><span class="o">.</span><span class="mi">3</span>
<span class="o">./</span><span class="n">configure</span><span class="w"> </span><span class="o">--</span><span class="n">prefix</span><span class="o">=/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span>
<span class="n">make</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">make</span><span class="w"> </span><span class="n">install</span>
<span class="n">ln</span><span class="w"> </span><span class="o">-</span><span class="n">s</span><span class="w"> </span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">spawn</span><span class="o">-</span><span class="n">fcgi</span><span class="w"> </span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span>
</code></pre></div>
<p>到这里我们就完成了Python的所有包安装,下面我们来部署nginx(本博有大量文章来介绍安装nginx,这里还是再来一遍吧..)</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>zlib-devel<span class="w"> </span>pcre-devel<span class="w"> </span>openssl-devel<span class="w"> </span><span class="c1"># 安装依赖</span>
wget<span class="w"> </span>http://nginx.org/download/nginx-1.0.13.tar.gz<span class="w"> </span><span class="c1"># 下载</span>
tar<span class="w"> </span>-zxvf<span class="w"> </span>nginx-1.0.13.tar.gz
<span class="nb">cd</span><span class="w"> </span>nginx-1.0.13
<span class="w"> </span>./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nginx<span class="se">\ </span><span class="w"> </span><span class="c1"># 指定安装目录为/usr/local/nginx</span>
--with-openssl<span class="o">=</span>/usr/include/openssl<span class="se">\ </span><span class="w"> </span><span class="c1"># 启用ssl</span>
--with-pcre<span class="se">\ </span><span class="w"> </span><span class="c1"># 启用正规表达式</span>
--with-http_stub_status_module<span class="w"> </span><span class="c1"># 安装可以查看nginx状态的程序</span>
make<span class="w"> </span><span class="o">&&</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>我们修改nginx的配置文件:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nt">location</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="p">{</span>
<span class="err">#</span><span class="w"> </span><span class="err">root</span><span class="w"> </span><span class="err">html</span><span class="p">;</span>
<span class="err">#</span><span class="w"> </span><span class="err">index</span><span class="w"> </span><span class="err">index.html</span><span class="w"> </span><span class="err">index.htm</span><span class="p">;</span>
<span class="w"> </span><span class="err">include</span><span class="w"> </span><span class="err">fastcgi_params</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_param</span><span class="w"> </span><span class="err">SCRIPT_FILENAME</span><span class="w"> </span><span class="err">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_param</span><span class="w"> </span><span class="err">PATH_INFO</span><span class="w"> </span><span class="err">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_pass</span><span class="w"> </span><span class="n">unix</span><span class="p">:</span><span class="o">/</span><span class="n">tmp</span><span class="o">/</span><span class="n">pyweb</span><span class="o">.</span><span class="n">sock</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_param</span><span class="w"> </span><span class="err">SERVER_ADDR</span><span class="w"> </span><span class="err">$server_addr</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_param</span><span class="w"> </span><span class="err">SERVER_PORT</span><span class="w"> </span><span class="err">$server_port</span><span class="p">;</span>
<span class="w"> </span><span class="err">fastcgi_param</span><span class="w"> </span><span class="err">SERVER_NAME</span><span class="w"> </span><span class="err">$server_name</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
</code></pre></div>
<p>然后创建一个web.py程序:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python</span>
<span class="c1">#-*-coding:utf8-*-</span>
<span class="kn">import</span> <span class="nn">web</span>
<span class="n">urls</span> <span class="o">=</span> <span class="p">(</span><span class="s2">"/.*"</span><span class="p">,</span> <span class="s2">"hello"</span><span class="p">)</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">web</span><span class="o">.</span><span class="n">application</span><span class="p">(</span><span class="n">urls</span><span class="p">,</span> <span class="nb">globals</span><span class="p">())</span>
<span class="k">class</span> <span class="nc">hello</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">GET</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s1">'Hello, world!'</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">web</span><span class="o">.</span><span class="n">wsgi</span><span class="o">.</span><span class="n">runwsgi</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">func</span><span class="p">,</span> <span class="n">addr</span> <span class="o">=</span> <span class="kc">None</span><span class="p">:</span> <span class="n">web</span><span class="o">.</span><span class="n">wsgi</span><span class="o">.</span><span class="n">runfcgi</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="n">addr</span><span class="p">)</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>将内容保存到/usr/local/nginx/html/下命名为index.py(或任意你喜欢的名字)
然后赋予其执行权限:</p>
<div class="highlight"><pre><span></span><code>chmod +x /usr/local/nginx/html/index.py
</code></pre></div>
<p>通过命令创建spawn-fcgi进程:</p>
<div class="highlight"><pre><span></span><code>spawn-fcgi -d /usr/local/nginx/html/ -f /usr/local/nginx/html/index.py -s /tmp/pyweb.sock -u nobody -g nobody
</code></pre></div>
<p>我们使用unix socket,并用nginx的用户来创建.现在我们访问http://192.168.3.3/就可以看到:<code>Hello, world!</code></p>lvs+keepalived实现高可用群集配置详解2012-04-16T14:25:00+08:002012-04-16T14:25:00+08:00coldtag:www.linuxzen.com,2012-04-16:/lvskeepalivedshi-xian-gao-ke-yong-qun-ji-pei-zhi-xiang-jie.html<p>lvs是一个开源的软件,由毕业于国防科技大学的章文嵩博士于1998年5月创立(中国人的项目),可以实现LINUX平台下的简单负载均衡。LVS是Linux Virtual Server的缩写,意思是Linux虚拟服务器。本文将介绍lvs结合keepalived实现一个高科用的Linux群集系统.</p>
<p>lvs有三种工作模式NAT(地址转换),IP Tunneling …</p><p>lvs是一个开源的软件,由毕业于国防科技大学的章文嵩博士于1998年5月创立(中国人的项目),可以实现LINUX平台下的简单负载均衡。LVS是Linux Virtual Server的缩写,意思是Linux虚拟服务器。本文将介绍lvs结合keepalived实现一个高科用的Linux群集系统.</p>
<p>lvs有三种工作模式NAT(地址转换),IP Tunneling(IP隧道)、Direct Routing(直接路由)。
工作效率最低的是NAT模式,但NAT模式可以用于各种系统,各种环境的负载均衡,只需要一个公网ip即可实现
IP Tunneling模式调度器将连接分发到不同的后端real server,然后由real server处理请求直接相应给用户,大大提高了调度器的调度效率,后端real server没有物理位置和逻辑关系的限制,后端real server可以在Lan/Wlan,但是后端real server必须支持IP隧道协议.
DR(Direct Routing)是效率最高的,与IP Tunneling类似,都是处理一般连接,将请求给后端real server,然后由real server处理请求直接相应给用户,Direct Routing与IP Tunneling相比,没有IP封装的开销,但由于采用物理层,所以DR模式的调度器和后端real server必须在一个物理网段里,中间不能过路由器(也就是一个交换机相连).</p>
<p>lvs支持8种不同的调度算法轮叫(rr)、加权轮叫(wrr)、最小连接(lc)、加权最小连接(wlc)、基于局部性最小连接(lblc)、带复制的基于局部性最少链接(lblcr)、目标地址散列(dh)和源地址散列(sh).</p>
<p>下面就介绍如何来安装和配置lvs+keepalived</p>
<p>本文使用环境:
操作系统:CentOS 5.5 32bit</p>
<p>主调度器:192.168.3.101/24</p>
<p>备调度器:192.168.3.102/24</p>
<p>后端real server: 192.168.3.3/24 |192.168.3.102/24(我们这里使用备用lvs作为一个测试</p>
<p>vip(virtual ip):192.168.3.100/24</p>
<p>lvs在2.6的内核中是默认支持的,所以我们就不需要在来安装,但是我们需要安装用户配置工具ipvsadm</p>
<div class="highlight"><pre><span></span><code>yum -y install ipvsadm # 分别在主从lvs上执行安装ipvsadm
</code></pre></div>
<p>我们查看lvs是否支持:</p>
<div class="highlight"><pre><span></span><code>lsmod ¦ grep ip_vs #
ip_vs 78081 1
modprobe -l¦ grep ip_vs
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_dh.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_ftp.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_lblc.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_lblcr.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_lc.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_nq.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_rr.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_sed.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_sh.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_wlc.ko
/lib/modules/2.6.18-194.el5/kernel/net/ipv4/ipvs/ip_vs_wrr.ko
</code></pre></div>
<p>本文介绍lvs的<code>DR</code>模式,首先部署keepalived.本博前面已经介绍如何来安装keepalived.这里就不在只简单的贴一下步骤:</p>
<p>在主备服务器上部署keepalived(因为前面已经rpm包安装了ipvsadm,所以就不需要重复安装):</p>
<div class="highlight"><pre><span></span><code>vi /etc/sysctl.conf
net.ipv4.ip_forward = 1 # 此参数改为1
sysctl -p # 使修改生效
</code></pre></div>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>openssl-devel
<span class="c1"># 下载并安装keepalived</span>
wget<span class="w"> </span>http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>keepalived-1.1.19.tar.gz
<span class="nb">cd</span><span class="w"> </span>keepalived-1.1.19
./configure<span class="w"> </span>--prefix<span class="o">=</span>/<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 安装在默认位置(配置文件,二进制文件,启动脚本放到默认位置)</span>
--mandir<span class="o">=</span>/usr/local/share/man/<span class="w"> </span><span class="se">\</span>
--with-kernel-dir<span class="o">=</span>/usr/src/kernels/2.6.18-194.el5-i686/<span class="w"> </span><span class="c1"># 需要内核的头文件</span>
make<span class="w"> </span><span class="o">&&</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>在主备lvs上安装keepalived完毕后我们先来配置主lvs上的keepalived:
编辑配置文件<code>/etc/keepalived/keepalived.conf</code>:</p>
<div class="highlight"><pre><span></span><code><span class="err">!</span><span class="w"> </span><span class="n">Configuration</span><span class="w"> </span><span class="k">File</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">keepalived</span>
<span class="n">global_defs</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">notification_email</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">coldnight</span><span class="nv">@linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">发生故障时发送的邮箱</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">notification_email_from</span><span class="w"> </span><span class="n">linuxzen</span><span class="nv">@linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">使用哪个邮箱发送</span>
<span class="w"> </span><span class="n">smtp_server</span><span class="w"> </span><span class="n">linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">发件服务器</span>
<span class="w"> </span><span class="n">smtp_connect_timeout</span><span class="w"> </span><span class="mi">30</span>
<span class="w"> </span><span class="n">router_id</span><span class="w"> </span><span class="n">LVS_DEVEL</span>
<span class="err">}</span>
<span class="n">vrrp_instance</span><span class="w"> </span><span class="n">VI_1</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="k">state</span><span class="w"> </span><span class="n">MASTER</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">标示为主lvs</span>
<span class="w"> </span><span class="n">interface</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">HA检测端口</span>
<span class="w"> </span><span class="n">virtual_router_id</span><span class="w"> </span><span class="mi">51</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">主备的virtual_router_id</span><span class="w"> </span><span class="n">必须相同</span>
<span class="w"> </span><span class="n">priority</span><span class="w"> </span><span class="mi">100</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">优先级</span><span class="p">,</span><span class="n">备lvs要比主lvs稍小</span>
<span class="w"> </span><span class="n">advert_int</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">VRRP</span><span class="w"> </span><span class="n">Multicast</span><span class="w"> </span><span class="n">广播周期秒数</span>
<span class="w"> </span><span class="n">authentication</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义认证</span>
<span class="w"> </span><span class="n">auth_type</span><span class="w"> </span><span class="n">PASS</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">认证方式为口令认证</span>
<span class="w"> </span><span class="n">auth_pass</span><span class="w"> </span><span class="mi">1111</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义口令</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">virtual_ipaddress</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义vip</span>
<span class="w"> </span><span class="mf">192.168.3.100</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">多个vip可换行添加</span>
<span class="w"> </span><span class="err">}</span>
<span class="err">}</span>
<span class="n">virtual_server</span><span class="w"> </span><span class="mf">192.168.3.100</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">delay_loop</span><span class="w"> </span><span class="mi">6</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">每隔6秒查看realserver状态</span>
<span class="w"> </span><span class="n">lb_algo</span><span class="w"> </span><span class="n">wlc</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">调度算法为加权最小连接数</span>
<span class="w"> </span><span class="n">lb_kind</span><span class="w"> </span><span class="n">DR</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">lvs工作模式为DR</span><span class="p">(</span><span class="n">直接路由</span><span class="p">)</span><span class="n">模式</span>
<span class="w"> </span><span class="n">nat_mask</span><span class="w"> </span><span class="mf">255.255.255.0</span>
<span class="w"> </span><span class="n">persistence_timeout</span><span class="w"> </span><span class="mi">50</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">同一IP</span><span class="w"> </span><span class="n">的连接50秒内被分配到同一台realserver</span><span class="p">(</span><span class="n">测试时建议改为0</span><span class="p">)</span>
<span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="n">TCP</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">用TCP监测realserver的状态</span>
<span class="w"> </span><span class="n">real_server</span><span class="w"> </span><span class="mf">192.168.3.3</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义realserver</span>
<span class="w"> </span><span class="n">weight</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义权重</span>
<span class="w"> </span><span class="n">TCP_CHECK</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">注意TCP_CHECK和</span><span class="err">{</span><span class="n">之间的空格</span><span class="p">,</span><span class="n">如果没有的话只会添加第一个realserver</span>
<span class="w"> </span><span class="n">connect_timeout</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">三秒无响应超时</span>
<span class="w"> </span><span class="n">nb_get_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">delay_before_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">connect_port</span><span class="w"> </span><span class="mi">80</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">real_server</span><span class="w"> </span><span class="mf">192.168.3.102</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">weight</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">TCP_CHECK</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">connect_timeout</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">nb_get_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">delay_before_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">connect_port</span><span class="w"> </span><span class="mi">80</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="err">}</span>
<span class="err">}</span>
</code></pre></div>
<p>配置备用lvs的keepalived,只需要将state MASTER 改为state BACKUP,降低priority 100 的值:</p>
<div class="highlight"><pre><span></span><code><span class="err">!</span><span class="w"> </span><span class="n">Configuration</span><span class="w"> </span><span class="k">File</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">keepalived</span>
<span class="n">global_defs</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">notification_email</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">coldnight</span><span class="nv">@linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">发生故障时发送的邮箱</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">notification_email_from</span><span class="w"> </span><span class="n">linuxzen</span><span class="nv">@linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">使用哪个邮箱发送</span>
<span class="w"> </span><span class="n">smtp_server</span><span class="w"> </span><span class="n">linuxzen</span><span class="p">.</span><span class="n">com</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">发件服务器</span>
<span class="w"> </span><span class="n">smtp_connect_timeout</span><span class="w"> </span><span class="mi">30</span>
<span class="w"> </span><span class="n">router_id</span><span class="w"> </span><span class="n">LVS_DEVEL</span>
<span class="err">}</span>
<span class="n">vrrp_instance</span><span class="w"> </span><span class="n">VI_1</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="k">state</span><span class="w"> </span><span class="k">BACKUP</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">标示为备lvs</span>
<span class="w"> </span><span class="n">interface</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">HA检测端口</span>
<span class="w"> </span><span class="n">virtual_router_id</span><span class="w"> </span><span class="mi">51</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">主备的virtual_router_id</span><span class="w"> </span><span class="n">必须相同</span>
<span class="w"> </span><span class="n">priority</span><span class="w"> </span><span class="mi">99</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">优先级</span><span class="p">,</span><span class="n">备lvs要比主lvs稍小</span>
<span class="w"> </span><span class="n">advert_int</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">VRRP</span><span class="w"> </span><span class="n">Multicast</span><span class="w"> </span><span class="n">广播周期秒数</span>
<span class="w"> </span><span class="n">authentication</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义认证</span>
<span class="w"> </span><span class="n">auth_type</span><span class="w"> </span><span class="n">PASS</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">认证方式为口令认证</span>
<span class="w"> </span><span class="n">auth_pass</span><span class="w"> </span><span class="mi">1111</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义口令</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">virtual_ipaddress</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义vip</span>
<span class="w"> </span><span class="mf">192.168.3.100</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">多个vip可换行添加</span>
<span class="w"> </span><span class="err">}</span>
<span class="err">}</span>
<span class="n">virtual_server</span><span class="w"> </span><span class="mf">192.168.3.100</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">delay_loop</span><span class="w"> </span><span class="mi">6</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">每隔6秒查看realserver状态</span>
<span class="w"> </span><span class="n">lb_algo</span><span class="w"> </span><span class="n">wlc</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">调度算法为加权最小连接数</span>
<span class="w"> </span><span class="n">lb_kind</span><span class="w"> </span><span class="n">DR</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">lvs工作模式为DR</span><span class="p">(</span><span class="n">直接路由</span><span class="p">)</span><span class="n">模式</span>
<span class="w"> </span><span class="n">nat_mask</span><span class="w"> </span><span class="mf">255.255.255.0</span>
<span class="w"> </span><span class="n">persistence_timeout</span><span class="w"> </span><span class="mi">50</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">同一IP</span><span class="w"> </span><span class="n">的连接50秒内被分配到同一台realserver</span>
<span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="n">TCP</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">用TCP监测realserver的状态</span>
<span class="w"> </span><span class="n">real_server</span><span class="w"> </span><span class="mf">192.168.3.3</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义realserver</span>
<span class="w"> </span><span class="n">weight</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">定义权重</span>
<span class="w"> </span><span class="n">TCP_CHECK</span><span class="w"> </span><span class="err">{</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">注意TCP_CHECK和</span><span class="err">{</span><span class="n">之间的空格</span><span class="p">,</span><span class="n">如果没有的话只会添加第一个realserver</span>
<span class="w"> </span><span class="n">connect_timeout</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">三秒无响应超时</span>
<span class="w"> </span><span class="n">nb_get_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">delay_before_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">connect_port</span><span class="w"> </span><span class="mi">80</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">real_server</span><span class="w"> </span><span class="mf">192.168.3.102</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">weight</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">TCP_CHECK</span><span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">connect_timeout</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">nb_get_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">delay_before_retry</span><span class="w"> </span><span class="mi">3</span>
<span class="w"> </span><span class="n">connect_port</span><span class="w"> </span><span class="mi">80</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="err">}</span>
<span class="err">}</span>
</code></pre></div>
<p>由于使用keepalived就不需要使用脚本来配置lvs调度器,但是这里我们还是会给出一个脚本内容,但我们不会用到这个脚本:lvs已经内置于内核,配置命令是ipvsadm,所以lvs的一些操作是通过ipvsadm来控制.下面我们就编写脚本来实现lvs的DR模式:</p>
<p>编写脚本lvsdr:</p>
<p>我们把lvs<code>vi /etc/init.d/lvsdr</code>添加如下内容</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="c1"># 定义虚拟ip</span>
<span class="nv">VIP</span><span class="o">=</span><span class="m">192</span>.168.3.100
<span class="c1"># 定义realserver,并已逗号分开</span>
<span class="nv">RIPS</span><span class="o">=</span><span class="m">192</span>.168.3.3,192.168.3.102<span class="w"> </span><span class="c1">#,192.168.3.5,192.168.3.6</span>
<span class="c1"># 定义提供服务的端口</span>
<span class="nv">SERVICE</span><span class="o">=</span><span class="m">80</span>
<span class="c1"># 调用init.d脚本的标准库</span>
.<span class="w"> </span>/etc/rc.d/init.d/functions
<span class="k">case</span><span class="w"> </span><span class="nv">$1</span><span class="w"> </span><span class="k">in</span>
<span class="w"> </span>start<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"Start LVS of DR Mode"</span>
<span class="w"> </span><span class="c1"># lvs dr模式不需要路由转发,但是keepalived需要</span>
<span class="w"> </span><span class="c1">#echo "0" > /proc/sys/net/ipv4/ip_forward</span>
<span class="w"> </span><span class="c1"># 开启icmp包重定向</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"1"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/all/send_redirects
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"1"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/default/send_redirects
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"1"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/eth0/send_redirects
<span class="w"> </span><span class="c1"># 绑定虚拟ip</span>
<span class="w"> </span>ifconfig<span class="w"> </span>eth0:0<span class="w"> </span><span class="nv">$VIP</span><span class="w"> </span>broadcast<span class="w"> </span><span class="nv">$VIP</span><span class="w"> </span>netmask<span class="w"> </span><span class="m">255</span>.255.255.255<span class="w"> </span>up
<span class="w"> </span>route<span class="w"> </span>add<span class="w"> </span>-host<span class="w"> </span><span class="nv">$VIP</span><span class="w"> </span>dev<span class="w"> </span>eth0:0
<span class="w"> </span><span class="c1"># 清除lvs规则</span>
<span class="w"> </span>ipvsadm<span class="w"> </span>-C
<span class="w"> </span><span class="c1"># 添加一条虚拟服务器记录</span>
<span class="w"> </span><span class="c1"># -p指定一定的时间内将相同的客户端分配到同一台后端服务器</span>
<span class="w"> </span><span class="c1"># 用于解决session的问题,测试时或有别的解决方案时建议去掉</span>
<span class="w"> </span>ipvsadm<span class="w"> </span>-A<span class="w"> </span>-t<span class="w"> </span><span class="nv">$VIP</span>:<span class="nv">$SERVICE</span><span class="w"> </span>-s<span class="w"> </span>wlc<span class="w"> </span>-p
<span class="w"> </span><span class="c1"># 添加真实服务器记录</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span>RIP<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span><span class="nb">echo</span><span class="w"> </span><span class="nv">$RIPS</span><span class="w"> </span>¦sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'s/,/\n/g'</span><span class="sb">`</span>
<span class="w"> </span><span class="k">do</span>
<span class="w"> </span>ipvsadm<span class="w"> </span>-a<span class="w"> </span>-t<span class="w"> </span><span class="nv">$VIP</span>:<span class="nv">$SERVICE</span><span class="w"> </span>-r<span class="w"> </span><span class="nv">$RIP</span>:<span class="nv">$SERVICE</span><span class="w"> </span>-g<span class="w"> </span>-w<span class="w"> </span><span class="m">1</span>
<span class="w"> </span><span class="k">done</span>
<span class="w"> </span><span class="c1"># 设置tcp tcpfin udp的超时连接值</span>
<span class="w"> </span>ipvsadm<span class="w"> </span>--set<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">120</span><span class="w"> </span><span class="m">300</span>
<span class="w"> </span>ipvsadm
<span class="w"> </span><span class="p">;;</span>
<span class="w"> </span>stop<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"Stop LVS DR"</span>
<span class="w"> </span>ifconfig<span class="w"> </span>eth0:0<span class="w"> </span>down
<span class="w"> </span>ipvsadm<span class="w"> </span>-C
<span class="w"> </span><span class="p">;;</span>
<span class="w"> </span>*<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"Usage:</span><span class="nv">$0</span><span class="s2"> {start ¦ stop}"</span>
<span class="w"> </span><span class="nb">exit</span><span class="w"> </span><span class="m">1</span>
<span class="k">esac</span>
</code></pre></div>
<p>编辑完毕保存退出,然后给这个脚本执行权限:</p>
<div class="highlight"><pre><span></span><code>chmod +x /etc/init.d/lvsdr
</code></pre></div>
<p>然后就可以通过service命令来启动lvs dr模式</p>
<div class="highlight"><pre><span></span><code>service lvsdr start
</code></pre></div>
<p>将这个脚本分别放到主备lvs的/etc/init.d/下,赋予执行权限.
我们真正需要的是realserver的脚本,下面我们来编写realserver脚本,同样放在/etc/init.d/下,编辑rs脚本:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/init.d/lvsrs
<span class="c1">#!/bin/sh</span>
<span class="nv">VIP</span><span class="o">=</span><span class="m">192</span>.168.3.100
.<span class="w"> </span>/etc/rc.d/init.d/functions
<span class="k">case</span><span class="w"> </span><span class="nv">$1</span><span class="w"> </span><span class="k">in</span>
<span class="w"> </span>start<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"lo:0 port starting"</span>
<span class="w"> </span><span class="c1"># 为了相应lvs调度器转发过来的包,需在本地lo接口上绑定vip</span>
<span class="w"> </span>ifconfig<span class="w"> </span>lo:0<span class="w"> </span><span class="nv">$VIP</span><span class="w"> </span>broadcast<span class="w"> </span><span class="nv">$VIP</span><span class="w"> </span>netmask<span class="w"> </span><span class="m">255</span>.255.255.255<span class="w"> </span>up
<span class="w"> </span><span class="c1"># 限制arp请求</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"1"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/lo/arp_ignore
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"2"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/lo/arp_announce
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"1"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/all/arp_ignore
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"2"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/all/arp_announce
<span class="w"> </span><span class="p">;;</span>
<span class="w"> </span>stop<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"lo:0 port closing"</span>
<span class="w"> </span>ifconfig<span class="w"> </span>lo:0<span class="w"> </span>down
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"0"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/lo/arp_ignore
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"0"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/lo/arp_announce
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"0"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/all/arp_ignore
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"0"</span><span class="w"> </span>><span class="w"> </span>/proc/sys/net/ipv4/conf/all/arp_announce
<span class="w"> </span><span class="p">;;</span>
<span class="w"> </span>*<span class="o">)</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"Usage: </span><span class="nv">$0</span><span class="s2"> {start ¦ stop}"</span>
<span class="w"> </span><span class="nb">exit</span><span class="w"> </span><span class="m">1</span>
<span class="k">esac</span>
</code></pre></div>
<p>给脚本赋予执行权限</p>
<div class="highlight"><pre><span></span><code>chmod +x /etc/init.d/lvsrs
</code></pre></div>
<p>并将这个脚本放到所有的realserver的/etc/init.d/下.下面开始测试:</p>
<p>先来确认下我们做的变动:主从lvs分别安装keepalived,并且在/etc/init.d/下添加了lvsdr脚本(不使用).</p>
<p>后端realserver分别在/etc/init.d/下添加了lvsrs脚本.我们先测试keepalived:</p>
<p>首先在主调度器上启动keepalived:</p>
<div class="highlight"><pre><span></span><code>service keepalived start
</code></pre></div>
<p>查看日志文件:</p>
<div class="highlight"><pre><span></span><code><span class="n">tail</span><span class="w"> </span><span class="o">-</span><span class="mi">50</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">message</span>
<span class="n">Mar</span><span class="w"> </span><span class="mi">21</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">29</span><span class="p">:</span><span class="mi">10</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">kernel</span><span class="p">:</span><span class="w"> </span><span class="n">device</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="n">left</span><span class="w"> </span><span class="n">promiscuous</span><span class="w"> </span><span class="n">mode</span>
<span class="n">Mar</span><span class="w"> </span><span class="mi">21</span><span class="w"> </span><span class="mi">22</span><span class="p">:</span><span class="mi">29</span><span class="p">:</span><span class="mi">10</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">kernel</span><span class="p">:</span><span class="w"> </span><span class="n">type</span><span class="o">=</span><span class="mi">1700</span><span class="w"> </span><span class="n">audit</span><span class="p">(</span><span class="mf">1332340150.598</span><span class="p">:</span><span class="mi">12</span><span class="p">):</span><span class="w"> </span><span class="n">dev</span><span class="o">=</span><span class="n">eth0</span><span class="w"> </span><span class="n">prom</span><span class="o">=</span><span class="mi">0</span><span class="w"> </span><span class="n">old_prom</span><span class="o">=</span><span class="mi">256</span><span class="w"> </span><span class="n">auid</span><span class="o">=</span><span class="mi">4294967295</span><span class="w"> </span><span class="n">ses</span><span class="o">=</span><span class="mi">4294967295</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived</span><span class="p">:</span><span class="w"> </span><span class="n">Starting</span><span class="w"> </span><span class="n">Keepalived</span><span class="w"> </span><span class="n">v1</span><span class="o">.</span><span class="mf">1.19</span><span class="w"> </span><span class="p">(</span><span class="mi">04</span><span class="o">/</span><span class="mi">16</span><span class="p">,</span><span class="mi">2012</span><span class="p">)</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.101</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">command</span><span class="w"> </span><span class="n">channel</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Opening</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="s1">'/etc/keepalived/keepalived.conf'</span><span class="o">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Configuration</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">using</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">8897</span><span class="w"> </span><span class="n">Bytes</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Using</span><span class="w"> </span><span class="n">LinkWatch</span><span class="w"> </span><span class="n">kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span><span class="o">...</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived</span><span class="p">:</span><span class="w"> </span><span class="n">Starting</span><span class="w"> </span><span class="n">Healthcheck</span><span class="w"> </span><span class="n">child</span><span class="w"> </span><span class="n">process</span><span class="p">,</span><span class="w"> </span><span class="n">pid</span><span class="o">=</span><span class="mi">5369</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived</span><span class="p">:</span><span class="w"> </span><span class="n">Starting</span><span class="w"> </span><span class="n">VRRP</span><span class="w"> </span><span class="n">child</span><span class="w"> </span><span class="n">process</span><span class="p">,</span><span class="w"> </span><span class="n">pid</span><span class="o">=</span><span class="mi">5370</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.101</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">command</span><span class="w"> </span><span class="n">channel</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">gratutious</span><span class="w"> </span><span class="n">ARP</span><span class="w"> </span><span class="n">shared</span><span class="w"> </span><span class="n">channel</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Opening</span><span class="w"> </span><span class="n">file</span><span class="w"> </span><span class="s1">'/etc/keepalived/keepalived.conf'</span><span class="o">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Configuration</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">using</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="mi">36512</span><span class="w"> </span><span class="n">Bytes</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Using</span><span class="w"> </span><span class="n">LinkWatch</span><span class="w"> </span><span class="n">kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span><span class="o">...</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">32</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP</span><span class="w"> </span><span class="n">sockpool</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="n">ifindex</span><span class="p">(</span><span class="mi">2</span><span class="p">),</span><span class="w"> </span><span class="n">proto</span><span class="p">(</span><span class="mi">112</span><span class="p">),</span><span class="w"> </span><span class="n">fd</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span><span class="mi">11</span><span class="p">)]</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">33</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Transition</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">MASTER</span><span class="w"> </span><span class="n">STATE</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">34</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Entering</span><span class="w"> </span><span class="n">MASTER</span><span class="w"> </span><span class="n">STATE</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">34</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">setting</span><span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="n">VIPs</span><span class="o">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">34</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">34</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Sending</span><span class="w"> </span><span class="n">gratuitous</span><span class="w"> </span><span class="n">ARPs</span><span class="w"> </span><span class="n">on</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">34</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">31</span><span class="p">:</span><span class="mi">39</span><span class="w"> </span><span class="k">master</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Sending</span><span class="w"> </span><span class="n">gratuitous</span><span class="w"> </span><span class="n">ARPs</span><span class="w"> </span><span class="n">on</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span>
</code></pre></div>
<p>然后在备用调度器上启动keepalived然后查看日志:</p>
<div class="highlight"><pre><span></span><code><span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Entering</span><span class="w"> </span><span class="k">BACKUP</span><span class="w"> </span><span class="k">STATE</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP</span><span class="w"> </span><span class="nl">sockpool</span><span class="p">:</span><span class="w"> </span><span class="o">[</span><span class="n">ifindex(2), proto(112), fd(11,12)</span><span class="o">]</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168.3.102</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Registering</span><span class="w"> </span><span class="n">Kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">command</span><span class="w"> </span><span class="n">channel</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Opening</span><span class="w"> </span><span class="k">file</span><span class="w"> </span><span class="s1">'/etc/keepalived/keepalived.conf'</span><span class="p">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Configuration</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="k">using</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="mi">8895</span><span class="w"> </span><span class="n">Bytes</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">kernel</span><span class="p">:</span><span class="w"> </span><span class="nl">IPVS</span><span class="p">:</span><span class="w"> </span><span class="o">[</span><span class="n">wlc</span><span class="o">]</span><span class="w"> </span><span class="n">scheduler</span><span class="w"> </span><span class="n">registered</span><span class="p">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="err">:</span><span class="mi">33</span><span class="err">:</span><span class="mi">35</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="nl">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="k">Using</span><span class="w"> </span><span class="n">LinkWatch</span><span class="w"> </span><span class="n">kernel</span><span class="w"> </span><span class="n">netlink</span><span class="w"> </span><span class="n">reflector</span><span class="p">...</span>
</code></pre></div>
<p>在主调度器上执行</p>
<div class="highlight"><pre><span></span><code>service keepalived stop
</code></pre></div>
<p>查看备用调度器日志:</p>
<div class="highlight"><pre><span></span><code><span class="n">tail</span><span class="w"> </span><span class="o">-</span><span class="mi">20</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">message</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">44</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Transition</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">MASTER</span><span class="w"> </span><span class="n">STATE</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">45</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Entering</span><span class="w"> </span><span class="n">MASTER</span><span class="w"> </span><span class="n">STATE</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">45</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">setting</span><span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="n">VIPs</span><span class="o">.</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">45</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">VRRP_Instance</span><span class="p">(</span><span class="n">VI_1</span><span class="p">)</span><span class="w"> </span><span class="n">Sending</span><span class="w"> </span><span class="n">gratuitous</span><span class="w"> </span><span class="n">ARPs</span><span class="w"> </span><span class="n">on</span><span class="w"> </span><span class="n">eth0</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">45</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_vrrp</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span><span class="w"> </span><span class="n">added</span>
<span class="n">Apr</span><span class="w"> </span><span class="mi">16</span><span class="w"> </span><span class="mi">13</span><span class="p">:</span><span class="mi">39</span><span class="p">:</span><span class="mi">45</span><span class="w"> </span><span class="n">slave</span><span class="w"> </span><span class="n">Keepalived_healthcheckers</span><span class="p">:</span><span class="w"> </span><span class="n">Netlink</span><span class="w"> </span><span class="n">reflector</span><span class="w"> </span><span class="n">reports</span><span class="w"> </span><span class="n">IP</span><span class="w"> </span><span class="mf">192.168</span><span class="o">.</span><span class="mf">3.100</span><span class="w"> </span><span class="n">added</span>
</code></pre></div>
<p>我们看到keepalived已经成功切换.</p>
<p>然后我们使用ipvsadm命令查看(在此之前要确认后端realserver已经启动了web服务):</p>
<div class="highlight"><pre><span></span><code><span class="n">ipvsadm</span>
<span class="n">IP</span><span class="w"> </span><span class="kr">Virtual</span><span class="w"> </span><span class="n">Server</span><span class="w"> </span><span class="n">version</span><span class="w"> </span><span class="mf">1.2.1</span><span class="w"> </span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="mi">4096</span><span class="p">)</span>
<span class="n">Prot</span><span class="w"> </span><span class="n">LocalAddress</span><span class="o">:</span><span class="n">Port</span><span class="w"> </span><span class="n">Scheduler</span><span class="w"> </span><span class="n">Flags</span>
<span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">RemoteAddress</span><span class="o">:</span><span class="n">Port</span><span class="w"> </span><span class="n">Forward</span><span class="w"> </span><span class="n">Weight</span><span class="w"> </span><span class="n">ActiveConn</span><span class="w"> </span><span class="n">InActConn</span>
<span class="n">TCP</span><span class="w"> </span><span class="mf">192.168.3.100</span><span class="o">:</span><span class="n">http</span><span class="w"> </span><span class="n">wlc</span>
<span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="mf">192.168.3.3</span><span class="o">:</span><span class="n">http</span><span class="w"> </span><span class="n">Route</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="mf">192.168.3.102</span><span class="o">:</span><span class="n">http</span><span class="w"> </span><span class="n">Route</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span>
</code></pre></div>
<p>然后分别启动后端realserver的lvsrs服务:</p>
<div class="highlight"><pre><span></span><code>servie lvsrs start
</code></pre></div>
<p>然后浏览器访问192.168.3.100,如果keepalived的persistence_timeout参数值为0,而且两个后端realserver是不同的内容,刷新就可以看到两个不同的页面交替.</p>使用Python进行web开发2012-04-13T16:39:00+08:002012-04-13T16:39:00+08:00coldtag:www.linuxzen.com,2012-04-13:/shi-yong-pythonjin-xing-webkai-fa.html<p>最近有一个小的web项目,想用喜爱都python,但是想到之前接触过都django我感觉一阵不寒而栗,为什么?Django的配置太过复杂,而且小项目不太适合MVC的开发模式,所以我将目光转向了web.py这个小型web框架 …</p><p>最近有一个小的web项目,想用喜爱都python,但是想到之前接触过都django我感觉一阵不寒而栗,为什么?Django的配置太过复杂,而且小项目不太适合MVC的开发模式,所以我将目光转向了web.py这个小型web框架,并且真正让我动心都是其官方网站上都一句话:"Django lets you write web apps in Django. TurboGears lets you write web apps in TurboGears. Web.py lets you write web apps in Python." — Adam Atlas</p>
<p>最近切换了Ubuntu替换了Win7系统,所以这里介绍下Ubuntu都安装web.py</p>
<h3 id="easy_install">安装easy_install</h3>
<div class="highlight"><pre><span></span><code>sudo apt-get install python-pip
</code></pre></div>
<h3 id="easy_installwebpy">使用easy_install安装web.py</h3>
<div class="highlight"><pre><span></span><code>sudo easy_install web.py
</code></pre></div>
<h3 id="_1">测试是否安装成功:</h3>
<p>在python shell中执行:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">web</span>
</code></pre></div>
<p>如果没有报错则web.py安装成功.
下面开始我们第一个hello,world</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">web</span>
<span class="n">urls</span> <span class="o">=</span> <span class="p">(</span><span class="s2">"/.*"</span><span class="p">,</span> <span class="s2">"hello"</span><span class="p">)</span> <span class="c1"># 指定任何url都指向hello类</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">web</span><span class="o">.</span><span class="n">application</span><span class="p">(</span><span class="n">urls</span><span class="p">,</span> <span class="nb">globals</span><span class="p">())</span> <span class="c1"># 绑定url</span>
<span class="c1"># 定义相应类</span>
<span class="k">class</span> <span class="nc">hello</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">GET</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s1">'Hello, world!'</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>然后保存为<code>hello.py</code>并运行它</p>
<div class="highlight"><pre><span></span><code>python hello.py
</code></pre></div>
<p>然后会看到输出:http://0.0.0.0:8080/</p>
<p>然后浏览器访问:http://localhost:8080即可看到
<code>Hello, world!</code>
我们第一个用python写的web程序就建立完成.</p>Linux下配置vim一键编译C/C++并执行2012-04-05T12:07:00+08:002012-04-05T12:07:00+08:00coldtag:www.linuxzen.com,2012-04-05:/linuxxia-pei-zhi-vimyi-jian-bian-yi-ccbing-zhi-xing.html<p>最近在学习C++,编辑器当然是vim,想在编辑的时候可以一键编译,于是自己写了一个小脚本配合vim来实现.由于刚开始学,所以对C …</p><p>最近在学习C++,编辑器当然是vim,想在编辑的时候可以一键编译,于是自己写了一个小脚本配合vim来实现.由于刚开始学,所以对C/C++的扩展名不太了解,所以只对.cpp .cc .c进行处理.</p>
<p>首先在/usr/bin/下创建compile脚本:</p>
<div class="highlight"><pre><span></span><code>vi /usr/bin/compile
</code></pre></div>
<p>添加如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1"># Filename : compile</span>
<span class="c1"># Describe : To compile c/c++</span>
<span class="c1"># Author : cold night(www.linuxzen.com)</span>
<span class="c1"># Version : 0.2</span>
<span class="c1"># Change : 增加终端着色 </span>
clear
<span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$#</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="o">]</span>
<span class="k">then</span>
<span class="w"> </span><span class="nv">filename</span><span class="o">=</span><span class="nv">$1</span>
<span class="w"> </span><span class="nv">outname</span><span class="o">=</span><span class="si">${</span><span class="nv">filename</span><span class="p">%</span><span class="se">\.</span><span class="p">*</span><span class="si">}</span>
<span class="w"> </span><span class="nv">typename</span><span class="o">=</span><span class="si">${</span><span class="nv">filename</span><span class="p">#*</span><span class="se">\.</span><span class="si">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">(</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span><span class="s2">"</span><span class="nv">$typename</span><span class="s2">"</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"cpp"</span><span class="w"> </span><span class="o">||</span><span class="w"> </span><span class="s2">"</span><span class="nv">$typename</span><span class="s2">"</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"cc"</span><span class="w"> </span><span class="o">)</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-n<span class="w"> </span><span class="s2">"Compiling..."</span>
<span class="w"> </span>g++<span class="w"> </span>-o<span class="w"> </span><span class="s2">"</span><span class="nv">$outname</span><span class="s2">"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$filename</span><span class="s2">"</span><span class="w"> </span><span class="m">2</span>><span class="w"> </span>/tmp/errinfo<span class="w"> </span>><span class="p">&</span><span class="m">2</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">" \033[32;1mSuccess!!!\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">"\033[1;44mRunning...\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span>./<span class="s2">"</span><span class="nv">$outname</span><span class="s2">"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">" \033[1;31mError!!!\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">"\033[1;44mError Info:\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span>cat<span class="w"> </span>/tmp/errinfo
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span><span class="k">elif</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="s2">"</span><span class="nv">$typename</span><span class="s2">"</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"c"</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"Compiling..."</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span>gcc<span class="w"> </span>-o<span class="w"> </span><span class="s2">"</span><span class="nv">$outname</span><span class="s2">"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$filename</span><span class="s2">"</span><span class="w"> </span><span class="m">2</span>>/tmp/errinfo<span class="w"> </span>><span class="p">&</span><span class="m">2</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">" \033[32;1mSuccess!!!\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">"\033[1;44mRunning...\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span>./<span class="s2">"</span><span class="nv">$outname</span><span class="s2">"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">" \033[1;31mError!!!\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span>-e<span class="w"> </span><span class="s2">"\033[1;44mError Info:\033[0m"</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span>cat<span class="w"> </span>/tmp/errinfo
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"------------------------"</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span><span class="k">fi</span>
<span class="k">else</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'Error: No intput filename'</span>
<span class="k">fi</span>
</code></pre></div>
<p>然后给脚本赋予执行权限</p>
<div class="highlight"><pre><span></span><code>chmod +x /usr/bin/compile
</code></pre></div>
<p>然后编辑vim配置文件:</p>
<div class="highlight"><pre><span></span><code>vi ~/.vimrc
</code></pre></div>
<p>然后添加下面内容:</p>
<div class="highlight"><pre><span></span><code><span class="c">" C++ complier</span>
autocmd <span class="nb">FileType</span> cpp map <span class="p"><</span>F8<span class="p">></span> <span class="p"><</span>Esc<span class="p">></span>:<span class="k">w</span><span class="p">!<</span>CR<span class="p">></span>:<span class="p">!</span><span class="k">compile</span> %<span class="p"><</span>CR<span class="p">></span>
autocmd <span class="nb">FileType</span> <span class="k">cc</span> map <span class="p"><</span>F8<span class="p">></span> <span class="p"><</span>Esc<span class="p">></span>:<span class="k">w</span><span class="p">!<</span>CR<span class="p">></span>:<span class="p">!</span><span class="k">compile</span> %<span class="p"><</span>CR<span class="p">></span>
autocmd <span class="nb">FileType</span> <span class="k">c</span> map <span class="p"><</span>F8<span class="p">></span> <span class="p"><</span>Esc<span class="p">></span>:<span class="k">w</span><span class="p">!<</span>CR<span class="p">></span>:<span class="p">!</span><span class="k">compile</span> %<span class="p"><</span>CR<span class="p">></span>
</code></pre></div>
<p>配置完毕我们就可以用vim编辑C/C++源文件的时候按F8就可以进行一键编译执行.</p>使用cacti构建监控系统2012-03-14T14:02:00+08:002012-03-14T14:02:00+08:00coldtag:www.linuxzen.com,2012-03-14:/shi-yong-cactigou-jian-jian-kong-xi-tong.html<p>Cacti是一套基于PHP,MySQL,SNMP及RRDTool开发的网络流量监测图形分析工具,它是通过 snmpget来获取数据,使用 RRDtool绘画图形,可以指定每一个用户能查看树状结构、host以及任何一张图,还可以与LDAP结合进行用户验证,同时也能自己增加模板,功能非常强大完 …</p><p>Cacti是一套基于PHP,MySQL,SNMP及RRDTool开发的网络流量监测图形分析工具,它是通过 snmpget来获取数据,使用 RRDtool绘画图形,可以指定每一个用户能查看树状结构、host以及任何一张图,还可以与LDAP结合进行用户验证,同时也能自己增加模板,功能非常强大完善。界面友好。</p>
<p>cacti是用php语言实现的一个软件,它的主要功能是用snmp服务获取数据,然后用rrdtool储存和更新数据,当用户需要查看数据的时候用rrdtool生成图表呈现给用户。因此,snmp和rrdtool是cacti的关键。Snmp关系着数据的收集,rrdtool关系着数据存储和图表的生成。</p>
<p>cacti依赖于PHP+MYSQL环境,前面的几篇文章已经详细讲解了如何在Linux部署LNMP环境,这篇文章我们就是用前面几章所搭建的环境,所以这里不再讲解如何搭建环境,如果您不会可以先看看前几篇文章.</p>
<p>下面就介绍如何来部署cacti.</p>
<p>首先介绍本文所使用的环境:
server:</p>
<p>系统为CentOS 5.5 32bit</p>
<p>ip:192.168.3.120</p>
<p>cacti:cacti-0.8.7i.tar.gz</p>
<p>cacti使用SNMP采集数据,首先安装snmp数据采集工具:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>lm_sensors<span class="w"> </span>net-snmp<span class="w"> </span>net-snmp-utils
</code></pre></div>
<p>同时cacti又依赖于rrdtool生成图表所以首先安装rrdtool:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.7.tar.gz
yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>cairo-devel<span class="w"> </span>glib2-devel<span class="w"> </span>pango-devel<span class="w"> </span>intltool<span class="w"> </span><span class="c1"># 安装rrdtool依赖</span>
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>下面就来下载cacti安装:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://www.cacti.net/downloads/cacti-0.8.7i.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>cacti-0.8.7i.tar.gz
cp<span class="w"> </span>-r<span class="w"> </span>cacti-0.8.7i<span class="w"> </span>/usr/local/nginx/html/cacti<span class="w"> </span><span class="c1"># 复制到html目录</span>
useradd<span class="w"> </span>cactiuser<span class="w"> </span>-M<span class="w"> </span>-s<span class="w"> </span>/sbin/nologin<span class="w"> </span><span class="c1"># 创建cacti用户</span>
chown<span class="w"> </span>-R<span class="w"> </span>cactiuser.cactiuser<span class="w"> </span>/usr/local/nginx/html/cacti/rra/<span class="w"> </span><span class="c1"># 改变属主和属组</span>
chown<span class="w"> </span>-R<span class="w"> </span>cactiuser.cactiuser<span class="w"> </span>/usr/local/nginx/html/cacti/log/
</code></pre></div>
<p>然后进入到数据库创建cacti数据和创建一个用户:</p>
<div class="highlight"><pre><span></span><code>create<span class="w"> </span>database<span class="w"> </span>cactidb<span class="w"> </span>default<span class="w"> </span>character<span class="w"> </span><span class="nb">set</span><span class="w"> </span>utf8<span class="p">;</span><span class="w"> </span><span class="c1">#创建数据库</span>
grant<span class="w"> </span>all<span class="w"> </span>on<span class="w"> </span>cactidb.*<span class="w"> </span>to<span class="w"> </span>cactiuser@localhost<span class="w"> </span>identified<span class="w"> </span>by<span class="w"> </span><span class="s1">'123456'</span><span class="p">;</span><span class="w"> </span><span class="c1"># 创建一个mysql用户</span>
use<span class="w"> </span>cactidb<span class="w"> </span><span class="c1"># 使用刚才创建的数据库</span>
<span class="nb">source</span><span class="w"> </span>/usr/local/nginx/html/cacti/cacti.sql<span class="w"> </span><span class="c1"># 导入cacti数据</span>
</code></pre></div>
<p>接下来我们编辑cacti配置文件/usr/local/nginx/html/cacti/include/config.php</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/local/nginx/html/cacti/
vi<span class="w"> </span>include/config.php
</code></pre></div>
<p>编辑下面内容:</p>
<div class="highlight"><pre><span></span><code><span class="nv">$database_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"mysql"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库类型</span>
<span class="nv">$database_default</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"cactidb"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库名字</span>
<span class="nv">$database_hostname</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"localhost"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库主机</span>
<span class="nv">$database_username</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"cactiuser"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库用户</span>
<span class="nv">$database_password</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"123456"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库密码</span>
<span class="nv">$database_port</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"3306"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 数据库端口</span>
<span class="nv">$database_ssl</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>false<span class="p">;</span>
</code></pre></div>
<p>然后修改nginx配置文件像下面:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>html<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.php<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>location<span class="w"> </span>~<span class="w"> </span><span class="se">\.</span>php$<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>fastcgi_pass<span class="w"> </span>unix:/var/run/php-fpm/php-fpm.sock<span class="p">;</span>
<span class="w"> </span>fastcgi_index<span class="w"> </span>index.php<span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SCRIPT_FILENAME<span class="w"> </span><span class="nv">$document_root</span>/<span class="nv">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span>include<span class="w"> </span>fastcgi_params<span class="p">;</span>
<span class="w"> </span>include<span class="w"> </span>fastcgi.conf<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
</code></pre></div>
<p>重启nginx</p>
<div class="highlight"><pre><span></span><code>pkill<span class="w"> </span>-9<span class="w"> </span>nginx
/usr/local/nginx/sbin/nginx
</code></pre></div>
<p>然后设置php时区,</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/local/nginx/html/cacti/
vi<span class="w"> </span>include/global_constants.php
</code></pre></div>
<p>在第二行添加</p>
<div class="highlight"><pre><span></span><code>date_default_timezone_set<span class="o">(</span><span class="s2">"Asia/Chongqing"</span><span class="o">)</span><span class="p">;</span>
</code></pre></div>
<p>接下来配置snmp,编辑/etc/snmp/snmpd.conf</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/snmp/snmpd.conf
</code></pre></div>
<p>然后找到41行将public改成一个较为复杂的名字:</p>
<div class="highlight"><pre><span></span><code>com2sec<span class="w"> </span>notConfigUser<span class="w"> </span>default<span class="w"> </span>public
</code></pre></div>
<p>然后找到62行</p>
<div class="highlight"><pre><span></span><code>access<span class="w"> </span>notConfigGroup<span class="w"> </span><span class="s2">""</span><span class="w"> </span>any<span class="w"> </span>noauth<span class="w"> </span>exact<span class="w"> </span>systemview<span class="w"> </span>none<span class="w"> </span>none
</code></pre></div>
<p>将systemview改成all:</p>
<div class="highlight"><pre><span></span><code>access<span class="w"> </span>notConfigGroup<span class="w"> </span><span class="s2">""</span><span class="w"> </span>any<span class="w"> </span>noauth<span class="w"> </span>exact<span class="w"> </span>all<span class="w"> </span>none<span class="w"> </span>none
</code></pre></div>
<p>然后去掉85行的注释:</p>
<div class="highlight"><pre><span></span><code>view<span class="w"> </span>all<span class="w"> </span>included<span class="w"> </span>.1<span class="w"> </span><span class="m">80</span>
</code></pre></div>
<p>保存配置文件后启动snmp</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>snmpd<span class="w"> </span>start
</code></pre></div>
<p>然后在浏览器里输入:http://192.168.3.120/cacti/
然后根据提示一步步安装,安装好后使用admin密码admin登录.如果点击graphs不能显示图像的话执行:</p>
<div class="highlight"><pre><span></span><code>php<span class="w"> </span>/usr/local/nginx/html/cacti/poller.php<span class="w"> </span><span class="c1"># nginx下不会自动生成*.rrd文件必须手动执行这条命令才会生成,Debug没报错,测试权限也没问题,不知道怎么回事,望知道的能告知小弟</span>
</code></pre></div>
<p>为了方便把这句加入到cron,执行:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>vixie-cron<span class="w"> </span><span class="c1">#安装</span>
service<span class="w"> </span>crond<span class="w"> </span>start
crontab<span class="w"> </span>-e
</code></pre></div>
<p>添加如下内容:</p>
<div class="highlight"><pre><span></span><code>*/5<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>/usr/bin/php<span class="w"> </span>/usr/local/nginx/html/cacti/poller.php
</code></pre></div>
<p>好了这时候我们就可以打开查看生成的图像了.
注意如果报如下错误:
Call to undefined function session_unregister()
将session_unregister('username') 改成
$_SESSION['username']='';</p>lnmp环境搭建完全手册(四)——lnmp搭建(源码安装)2012-03-13T13:13:00+08:002012-03-13T13:13:00+08:00coldtag:www.linuxzen.com,2012-03-13:/lnmphuan-jing-da-jian-wan-quan-shou-ce-si-lnmpda-jian-yuan-ma-an-zhuang.html<p>上面3篇我们主要对系统进行了安装和配置,并且配置了yum包管理器,安装了几个常用的工具,这篇我们就来介绍如何来搭建lnmp环境.这里的LNMP环境是指Linux下搭建Nginx+MySQL+PHP.Nginx是一个高性能的 HTTP 和 反向代理 服务器,也 …</p><p>上面3篇我们主要对系统进行了安装和配置,并且配置了yum包管理器,安装了几个常用的工具,这篇我们就来介绍如何来搭建lnmp环境.这里的LNMP环境是指Linux下搭建Nginx+MySQL+PHP.Nginx是一个高性能的 HTTP 和 反向代理 服务器,也是一个 IMAP/POP3/SMTP 代理服务器。Nginx不仅可以作为web服务器,也可以作为负载均衡器,之前也有文章介绍,大家可以看一下.</p>
<p>MySQL是一款开源免费的数据软件,MySQL是一个小型关系型数据库管理系统,其体积小、速度快、总体拥有成本低,尤其是开放源码这一特点,许多中小型网站为了降低网站总体拥有成本而选择了MySQL作为网站数据库.</p>
<p>PHP,是英文超级文本预处理语言Hypertext Preprocessor的缩写。PHP 是一种 HTML 内嵌式的语言,是一种在服务器端执行的嵌入HTML文档的脚本语言,语言的风格有类似于C语言,被广泛的运用。</p>
<p>nginx当前最新稳定版是nginx-1.0.13
首先我们下载nginx,在Linux下执行下面命令:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/src<span class="w"> </span><span class="c1"># 一般软件源码放在这个目录下</span>
wget<span class="w"> </span>http://nginx.org/download/nginx-1.0.13.tar.gz<span class="w"> </span><span class="c1"># 下载</span>
nginx会有几个依赖包,我们首先安装依赖,不要安装过程中会报错:
yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>zlib-devel<span class="w"> </span>pcre-devel<span class="w"> </span>openssl-devel
</code></pre></div>
<p>一般源代码安装分4个步骤(有人也会省去第一个步骤),<code>解压</code>(tar命令)<code>预编译</code>(执行源码包下的configure),<code>编译</code>(make),<code>编译安装</code>(make install)
首先我们解压源码包:</p>
<div class="highlight"><pre><span></span><code>tar -zxvf nginx-1.0.13.tar.gz
</code></pre></div>
<p>这里解释下加压参数,z代表gzip(也就是后面的.gz文件)x代表加压,v表示显示详细信息,-f使用档案文件或设备(必选参数)</p>
<p>然后我们进行预编译,一般预编译会带上一些参数,已达到我们想要安装的效果,比如启用某个功能,禁用某个功能:</p>
<p>进入源码包目录进行预编译:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>nginx-1.0.13
<span class="w"> </span>./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nginx<span class="se">\ </span><span class="w"> </span><span class="c1"># 指定安装目录为/usr/local/nginx</span>
--with-openssl<span class="o">=</span>/usr/include/openssl<span class="se">\ </span><span class="w"> </span><span class="c1"># 启用ssl</span>
--with-pcre<span class="se">\ </span><span class="w"> </span><span class="c1"># 启用正规表达式</span>
--with-http_stub_status_module<span class="w"> </span><span class="c1"># 安装可以查看nginx状态的程序</span>
</code></pre></div>
<p>其中./configure指执行当前目录下的<code>configure</code>文件</p>
<p>预编译完成后我们就可以进行编译和安装:</p>
<div class="highlight"><pre><span></span><code>make #编译
</code></pre></div>
<p>执行后make后会有大量输出,等待输出完成后如果没有报错就可以进行安装执行:</p>
<div class="highlight"><pre><span></span><code>make install #安装
</code></pre></div>
<p>安装完成后我们可以到相应的目录查看安装的文件:</p>
<div class="highlight"><pre><span></span><code>ls<span class="w"> </span>/usr/local/nginx/
conf<span class="w"> </span>html<span class="w"> </span>logs<span class="w"> </span>sbin
</code></pre></div>
<p>好了,下面我们启动nginx:</p>
<div class="highlight"><pre><span></span><code>/usr/local/nginx/sbin/nginx
</code></pre></div>
<p>通过查看端口看nginx是否启动成功,nginx占用TCP的80端口,执行下面命令:</p>
<div class="highlight"><pre><span></span><code> netstat -antlp ¦ grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5946/nginx
</code></pre></div>
<p>我们查看80端口是开放的</p>
<p>然后打开浏览器访问http://192.168.3.120,我们会看到Welcome to nginx(之前的版本是 It's Work):</p>
<p>nginx安装完毕后我们来安装MySQL ,我们使用MySQl-5.0.95版首先下载:</p>
<div class="highlight"><pre><span></span><code><span class="n">wget</span><span class="w"> </span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">dev</span><span class="o">.</span><span class="n">mysql</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">get</span><span class="o">/</span><span class="n">Downloads</span><span class="o">/</span><span class="n">MySQL</span><span class="o">-</span><span class="mf">5.0</span><span class="o">/</span><span class="n">mysql</span><span class="o">-</span><span class="mf">5.0</span><span class="o">.</span><span class="mf">95.</span><span class="n">tar</span><span class="o">.</span><span class="n">gz</span><span class="o">/</span><span class="n">from</span><span class="o">/</span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">mysql</span><span class="o">.</span><span class="n">cdpa</span><span class="o">.</span><span class="n">nsysu</span><span class="o">.</span><span class="n">edu</span><span class="o">.</span><span class="n">tw</span><span class="o">/</span>
</code></pre></div>
<p>安装之前我们先做一些准备工作,</p>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum -y install ncurses-devel
</code></pre></div>
<p>创建MySQL用户:</p>
<div class="highlight"><pre><span></span><code>useradd<span class="w"> </span>-M<span class="w"> </span>-s<span class="w"> </span>/sbin/nologin<span class="w"> </span>mysql<span class="w"> </span><span class="c1"># -M不创建home目录,-s指定shell为不登录</span>
</code></pre></div>
<p>然后进行安装:</p>
<div class="highlight"><pre><span></span><code>tar<span class="w"> </span>-zxvf<span class="w"> </span>mysql-5.0.95.tar.gz
<span class="nb">cd</span><span class="w"> </span>mysql-5.0.95
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/mysql<span class="w"> </span><span class="se">\</span>
--without-debug<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 取消调试模式提高性能</span>
--with-extra-charsets<span class="o">=</span>utf8,gbk<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 仅仅指定需要的默认字符集提高性能</span>
--enable-assembler<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 使用汇编模式提高性能</span>
--with-mysqld-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 以静态方式编译提高性能</span>
--with-client-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\</span>
--with-unix-socket-path<span class="o">=</span>/tmp/mysql.sock<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 使用unix socket提高性能</span>
--with-ssl
make
make<span class="w"> </span>install
</code></pre></div>
<p>安装完成后复制配置文件和启动脚本:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>support-files/my-medium.cnf<span class="w"> </span>/etc/my.cnf<span class="w"> </span><span class="c1"># 复制配置文件</span>
cp<span class="w"> </span>support-files/mysql.server<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span><span class="c1"># 复制启动脚本</span>
chmod<span class="w"> </span>+x<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span><span class="c1"># 给启动脚本执行权限</span>
</code></pre></div>
<p>为了以后方便我们为所有的二进制可执行文件和动态链接库文件做一个软连接:</p>
<div class="highlight"><pre><span></span><code>ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/mysql/bin/*<span class="w"> </span>/usr/local/bin/<span class="w"> </span><span class="c1"># 为可执行的二进制文件做软连接</span>
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/mysql/lib/mysql/lib*<span class="w"> </span>/usr/lib/<span class="w"> </span><span class="c1"># 为动态链接库做一个软连接</span>
</code></pre></div>
<p>然后我们初始化数据库:</p>
<div class="highlight"><pre><span></span><code>mysql_install_db<span class="w"> </span>--user<span class="o">=</span>mysql<span class="w"> </span><span class="c1"># 用MySQL用户安装数据库</span>
</code></pre></div>
<p>为了MySQL能正常使用我们需要更改一下MySQL安装目录和MySQL的数据库目录的属主和属组:</p>
<div class="highlight"><pre><span></span><code>chown<span class="w"> </span>-R<span class="w"> </span>root.mysql<span class="w"> </span>/usr/local/mysql/<span class="w"> </span><span class="c1"># 更改安装目录属主为root,属组为mysql</span>
chown<span class="w"> </span>-R<span class="w"> </span>mysql.mysql<span class="w"> </span>/usr/local/mysql/var/<span class="w"> </span><span class="c1"># 更改数据库目录属主和属组都为mysql</span>
</code></pre></div>
<p>这里的<code>-R</code>参数用来应用到所有子目录和文件</p>
<p>配置完毕后我们启动mysql:</p>
<div class="highlight"><pre><span></span><code>service mysqld start
</code></pre></div>
<p>现在我们查看MySQL是否启动成功,MySQL占用TCP的3306端口,我们查看端口是否被占用:</p>
<div class="highlight"><pre><span></span><code>netstat -antlp ¦ grep 3306
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 32143/mysqld
</code></pre></div>
<p>然后我们通过mysql命令来连接mysql:</p>
<div class="highlight"><pre><span></span><code>mysql
</code></pre></div>
<p>会显示如下内容表示已经成功启动MySQL并已经连接上</p>
<div class="highlight"><pre><span></span><code><span class="n">Welcome</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">MySQL</span><span class="w"> </span><span class="n">monitor</span><span class="p">.</span><span class="w"> </span><span class="n">Commands</span><span class="w"> </span><span class="k">end</span><span class="w"> </span><span class="n">with</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">or</span><span class="w"> </span><span class="n">\g.</span>
<span class="n">Your</span><span class="w"> </span><span class="n">MySQL</span><span class="w"> </span><span class="n">connection</span><span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="n">is</span><span class="w"> </span><span class="mh">1</span>
<span class="n">Server</span><span class="w"> </span><span class="nl">version:</span><span class="w"> </span><span class="mf">5.0.95</span><span class="o">-</span><span class="n">log</span><span class="w"> </span><span class="n">Source</span><span class="w"> </span><span class="n">distribution</span>
<span class="n">Copyright</span><span class="w"> </span><span class="p">(</span><span class="n">c</span><span class="p">)</span><span class="w"> </span><span class="mh">2000</span><span class="p">,</span><span class="w"> </span><span class="mh">2011</span><span class="p">,</span><span class="w"> </span><span class="n">Oracle</span><span class="w"> </span><span class="k">and</span><span class="o">/</span><span class="k">or</span><span class="w"> </span><span class="n">its</span><span class="w"> </span><span class="n">affiliates</span><span class="p">.</span><span class="w"> </span><span class="n">All</span><span class="w"> </span><span class="n">rights</span><span class="w"> </span><span class="n">reserved</span><span class="p">.</span>
<span class="n">Oracle</span><span class="w"> </span><span class="n">is</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">registered</span><span class="w"> </span><span class="n">trademark</span><span class="w"> </span><span class="n">of</span><span class="w"> </span><span class="n">Oracle</span><span class="w"> </span><span class="n">Corporation</span><span class="w"> </span><span class="k">and</span><span class="o">/</span><span class="k">or</span><span class="w"> </span><span class="n">its</span>
<span class="n">affiliates</span><span class="p">.</span><span class="w"> </span><span class="n">Other</span><span class="w"> </span><span class="n">names</span><span class="w"> </span><span class="n">may</span><span class="w"> </span><span class="n">be</span><span class="w"> </span><span class="n">trademarks</span><span class="w"> </span><span class="n">of</span><span class="w"> </span><span class="n">their</span><span class="w"> </span><span class="n">respective</span>
<span class="n">owners</span><span class="p">.</span>
<span class="n">Type</span><span class="w"> </span><span class="mh">'he</span><span class="n">lp</span><span class="p">;'</span><span class="w"> </span><span class="k">or</span><span class="w"> </span><span class="sc">'\h'</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">help</span><span class="p">.</span><span class="w"> </span><span class="n">Type</span><span class="w"> </span><span class="sc">'\c'</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="n">clear</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">current</span><span class="w"> </span><span class="k">input</span><span class="w"> </span><span class="n">statement</span><span class="p">.</span>
<span class="n">mysql</span><span class="o">></span>
</code></pre></div>
<p>MySQL安装完毕下面我们就来安装PHP,安装PHP前首先要安装几个源码包依赖:<code>libmcrypt</code> <code>mhash</code> <code>mcrypt</code></p>
<p>首先来安装几个源码包依赖:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://sourceforge.net/projects/mcrypt/files/Libmcrypt/2.5.8/libmcrypt-2.5.8.tar.bz2/download
tar<span class="w"> </span>-jxvf<span class="w"> </span>libmcrypt-2.5.8.tar.bz2<span class="w"> </span><span class="c1"># 这个包是bz2的 使用-j参数解压</span>
<span class="nb">cd</span><span class="w"> </span>libmcrypt-2.5.8
./configure
make
make<span class="w"> </span>install
<span class="c1">####################################################</span>
wget<span class="w"> </span>http://sourceforge.net/projects/mhash/files/mhash/0.9.9.9/mhash-0.9.9.9.tar.bz2/download
tar<span class="w"> </span>-jxvf<span class="w"> </span>mhash-0.9.9.9.tar.bz2
<span class="nb">cd</span><span class="w"> </span>mhash-0.9.9.9
./configure
make
make<span class="w"> </span>install
<span class="c1"># 这两个包安装完成后要把动态链接库做一个软连接到/usr/lib,以为接下来的mcrypt依赖于这两个包</span>
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/lib/libmcrypt*<span class="w"> </span>/usr/lib
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/lib/libmhash.*<span class="w"> </span>/usr/lib/
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/bin/libmcrypt-config<span class="w"> </span>/usr/bin/libmcrypt-config
<span class="c1">###########################################################</span>
wget<span class="w"> </span>http://sourceforge.net/projects/mcrypt/files/MCrypt/2.6.8/mcrypt-2.6.8.tar.gz/download
tar<span class="w"> </span>-zxvf<span class="w"> </span>mcrypt-2.6.8.tar.gz
<span class="nb">cd</span><span class="w"> </span>mcrypt-2.6.8
./configure
make
make<span class="w"> </span>install
</code></pre></div>
<p>然后下载php:</p>
<div class="highlight"><pre><span></span><code>wget http://cn2.php.net/get/php-5.4.0.tar.bz2/from/this/mirror
</code></pre></div>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum –y install libxml2-devel curl-devel libpng-devel openldap-devel
</code></pre></div>
<p>我们使用nginx调用php的时候使用fpm的方式,在php 5.4中加入了对php-fpm的支持,所以就不需要打补丁了.安装PHP:</p>
<div class="highlight"><pre><span></span><code>tar -jxvf php-5.4.0.tar.bz2
cd php-5.4.0
./configure --prefix=/usr/local/php --with-mysql=/usr/local/mysql/ --with-zlib --enable-xml --disable-rpath --enable-safe-mode --enable-bcmath --enable-shmop --enable-sysvsem --with-curl --with-curlwrappers --enable-fpm --enable-fastcgi --with-mcrypt --with-gd --with-openssl --with-mhash --enable-sockets --with-ldap --with-ldap-sasl --with-xmlrpc -enable-zip --enable-soap
make
make install
</code></pre></div>
<p>到这里整个LNMP已经安装完成.下面我们就配置php和nginx能运行php网站:
首先为php创建配置文件:</p>
<div class="highlight"><pre><span></span><code>cp php.ini-production /usr/local/php/php.ini # 如果是开发就复制php.ini-development
cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf
ln -s /usr/local/php/bin/php /usr/bin/
</code></pre></div>
<p>配置php-fpm,编辑php-fpm.conf</p>
<div class="highlight"><pre><span></span><code>vi /usr/local/php/etc/php-fpm.conf
</code></pre></div>
<p>找到listen那一行,修改成如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="n">listen</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span><span class="o">.</span><span class="n">sock</span><span class="w"> </span><span class="c1"># 使用unix socket</span>
</code></pre></div>
<p>启动php-fpm</p>
<div class="highlight"><pre><span></span><code><span class="n">mkdir</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span>
<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">php</span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span>
</code></pre></div>
<p>然后配置nginx,编辑nginx配置文件</p>
<div class="highlight"><pre><span></span><code>vi /usr/local/nginx/conf/nginx.conf
</code></pre></div>
<p>修改nginx配置文件支持php:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="n">server</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">listen</span><span class="w"> </span><span class="mi">80</span><span class="p">;</span>
<span class="w"> </span><span class="n">server_name</span><span class="w"> </span><span class="n">localhost</span><span class="p">;</span>
<span class="w"> </span><span class="c1">#charset koi8-r;</span>
<span class="w"> </span><span class="c1">#access_log logs/host.access.log main;</span>
<span class="w"> </span><span class="n">location</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="n">html</span><span class="p">;</span>
<span class="w"> </span><span class="n">index</span><span class="w"> </span><span class="n">index</span><span class="o">.</span><span class="n">php</span><span class="w"> </span><span class="n">index</span><span class="o">.</span><span class="n">html</span><span class="w"> </span><span class="n">index</span><span class="o">.</span><span class="n">htm</span><span class="p">;</span><span class="w"> </span><span class="c1"># 添加index.php的首页文件</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="c1"># 添加下面内容</span>
<span class="w"> </span><span class="n">location</span><span class="w"> </span><span class="o">~</span><span class="w"> </span>\<span class="o">.</span><span class="n">php</span><span class="o">$</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">fastcgi_pass</span><span class="w"> </span><span class="n">unix</span><span class="p">:</span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span><span class="o">/</span><span class="n">php</span><span class="o">-</span><span class="n">fpm</span><span class="o">.</span><span class="n">sock</span><span class="p">;</span>
<span class="w"> </span><span class="n">fastcgi_index</span><span class="w"> </span><span class="n">index</span><span class="o">.</span><span class="n">php</span><span class="p">;</span>
<span class="w"> </span><span class="n">fastcgi_param</span><span class="w"> </span><span class="n">SCRIPT_FILENAME</span><span class="w"> </span><span class="o">$</span><span class="n">document_root</span><span class="o">/$</span><span class="n">fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span><span class="n">include</span><span class="w"> </span><span class="n">fastcgi_params</span><span class="p">;</span>
<span class="w"> </span><span class="n">include</span><span class="w"> </span><span class="n">fastcgi</span><span class="o">.</span><span class="n">conf</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
</code></pre></div>
<p>修改完毕后保存退出重启nginx:</p>
<div class="highlight"><pre><span></span><code>pkill -9 nignx
/usr/local/nginx/sbin/nginx
</code></pre></div>
<p>然后在/usr/local/nginx/html下创建index.php,</p>
<div class="highlight"><pre><span></span><code>vi /usr/local/nginx/html/index.php
</code></pre></div>
<p>添加下面内容:</p>
<div class="highlight"><pre><span></span><code><span class="cp"><?php</span>
<span class="nb">phpinfo</span><span class="p">();</span>
<span class="cp">?></span>
</code></pre></div>
<p>保存退出后访问<a href="">http://192.168.3.120/index.php</a>,看到下面页面表示已经安装配置成功:</p>
<p><img alt="linuxzen.com" src="/upload/QQ截图20120313133251.jpg">
到这里我们的LNMP环境就完全搭建成功,运行你的网站或者学习你的PHP吧.</p>lnmp环境搭建完全手册(三)——应用安装(yum配置和包安装)2012-03-13T10:06:00+08:002012-03-13T10:06:00+08:00coldtag:www.linuxzen.com,2012-03-13:/lnmphuan-jing-da-jian-wan-quan-shou-ce-san-ying-yong-an-zhuang-yumpei-zhi-he-bao-an-zhuang.html<p>上两篇文章我给大家讲解了系统的安装,网络和ssh的配置和连接,这篇就给大家讲一下Linux如何安装包.CentOS使用rpm包管理,rpm包安装使用rpm命令</p>
<p>比如你有一个包为:</p>
<div class="highlight"><pre><span></span><code>abc-0.1.3-el5.centos.rpm
</code></pre></div>
<p>包的名称 …</p><p>上两篇文章我给大家讲解了系统的安装,网络和ssh的配置和连接,这篇就给大家讲一下Linux如何安装包.CentOS使用rpm包管理,rpm包安装使用rpm命令</p>
<p>比如你有一个包为:</p>
<div class="highlight"><pre><span></span><code>abc-0.1.3-el5.centos.rpm
</code></pre></div>
<p>包的名称由3部分组成,第一部分是包名,第二部分是版本号,第三部分是使用平台.如果想安装这个包使用如下命令安装:</p>
<div class="highlight"><pre><span></span><code>rpm<span class="w"> </span>-ivh<span class="w"> </span>abc-0.1.3-el5.centos.rpm<span class="w"> </span><span class="c1"># i是安装 v是显示详细信息,-h显示安装进度</span>
</code></pre></div>
<p>卸载这个包使用:</p>
<div class="highlight"><pre><span></span><code>rpm<span class="w"> </span>-e<span class="w"> </span>abc
</code></pre></div>
<p>卸载只要指定包名即可卸载.</p>
<p>因为Linux下的包依赖关系很复杂,一个包可能会有很多的依赖包,必须把所有依赖包都装了才能安装这个包.所以使用rpm安装包,就变得很麻烦很头疼.如果想工作变的轻松,我们可以借助工具,在rhel5发行的时候就集成了一个工具yum.Yum(全称为 Yellow dog Updater, Modified)是一个在Fedora和RedHat以及SUSE、CentOS中的Shell前端软件包管理器。基於RPM包管理,能够从指定的服务器自动下载RPM包并且安装,可以自动处理依赖性关系,并且一次安装所有依赖的软体包,无须繁琐地一次次下载、安装。</p>
<p>yum的使用需要指定一个yum仓库,我们首先来配置一个yum仓库,yum仓库可以是internet上提供的,但是rhel的yum库是收费的,但是我们用的CentOS的yum库是免费的.yum仓库还可以使用本地的yum库.我们这里使用本地yum库.资源就是使用我们的安装DVD盘.首先将我们的DVD盘放到虚拟光驱,然后在Linux下挂载光盘.
右键虚拟机标签->Setting(设置),选择CD/DVD(IDE),在右边点选Use ISO image file,然后点"Browse..",浏览找到CentOS 5.5 安装DVD盘.然后选择右上角的connect:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120311164241.jpg">
<img alt="wwww.linuxzen.com" src="/upload/QQ截图20120311164417.jpg"></p>
<p>然后ok保存,在终端或在你的ssh客户端上输入下面命令:</p>
<div class="highlight"><pre><span></span><code>mount<span class="w"> </span>/dev/hdc<span class="w"> </span>/mnt<span class="w"> </span><span class="c1"># 将光盘挂载到/mnt下</span>
</code></pre></div>
<p>然后修改yum配置文件,首先我们不是用默认提供的官方源,我们把CentOS-Base.repo重命名一下:</p>
<div class="highlight"><pre><span></span><code>mv<span class="w"> </span>/etc/yum.repos.d/CentOS-Base.repo<span class="w"> </span>/etc/yum.repos.d/CentOS-Base.repo.bak
</code></pre></div>
<p>然后修改本地源配置文件:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/yum.repos.d/CentOS-Media.repo
</code></pre></div>
<p>按i进入输入模式,修改成如下内容</p>
<div class="highlight"><pre><span></span><code><span class="o">[</span>c5-media<span class="o">]</span>
<span class="nv">name</span><span class="o">=</span>CentOS-<span class="nv">$releasever</span><span class="w"> </span>-<span class="w"> </span>Media
<span class="nv">baseurl</span><span class="o">=</span>file:///mnt<span class="w"> </span><span class="c1"># 源为挂载光盘的目录/mnt下</span>
<span class="c1"># file:///media/CentOS/</span>
<span class="c1"># file:///media/cdrom/</span>
<span class="c1"># file:///media/cdrecorder/</span>
<span class="nv">gpgcheck</span><span class="o">=</span><span class="m">1</span>
<span class="nv">enabled</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="c1"># 改为1启用这个源</span>
<span class="nv">gpgkey</span><span class="o">=</span>file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">修改完毕按</span><span class="n n-Quoted">`Esc`</span><span class="n">进入命令模式</span><span class="p">,</span><span class="n">输入</span><span class="n n-Quoted">`:`</span><span class="n">进入末行模式</span><span class="p">,</span><span class="n">在末行模式输入</span><span class="n n-Quoted">`wq`</span><span class="n">保存退出</span><span class="p">.</span>
<span class="n">修改完毕后执行</span><span class="o">:</span>
<span class="n n-Quoted">`</span><span class="n n-Quoted n-Quoted-Escape">``</span><span class="n n-Quoted">bash</span>
<span class="n n-Quoted">yum update</span>
</code></pre></div>
<p>会看到类似下面的输入,表示我们的配置是成功的:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120313095321.jpg"></p>
<p>配置完成后我们用他来安装几个常用的工具,</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>setuptool<span class="w"> </span>lrzsz<span class="w"> </span>wget<span class="w"> </span>gcc<span class="w"> </span>gcc-c++<span class="w"> </span>make<span class="w"> </span>which<span class="w"> </span>bzip2<span class="w"> </span>groff<span class="w"> </span>man
</code></pre></div>
<p>来介绍一下这几个工具,setuptool可以在终端界面下提供一个类似安装界面的图形化配置界面.
lrzsz可以通过其提供的命令rz/sz过ssh客户端对服务器进行上传和下载
wget是Linux下优秀的下载工具(http/ftp)
gcc是一款C/C++的编译器
make使用gcc根据makefile进行编译源代码
which是一个查找命令的工具
bzip2是一个压缩软件一般供tar调用
man是一个查看手册命令,如果碰到不懂的命令可以man一下.比如你不知道wget命令的用法可以:</p>
<div class="highlight"><pre><span></span><code>man<span class="w"> </span>wget
</code></pre></div>
<p>然后man命令就给提供这个命令的详细手册.好了一些准备工作到这里已经做好了.接下来的文章我们就介绍安装nginx.</p>lnmp环境搭建完全手册(二)——系统配置(实现虚拟机上网/ssh/)2012-03-10T11:31:00+08:002012-03-10T11:31:00+08:00coldtag:www.linuxzen.com,2012-03-10:/lnmphuan-jing-da-jian-wan-quan-shou-ce-er-xi-tong-pei-zhi-shi-xian-xu-ni-ji-shang-wang-ssh.html<p>上一篇介绍了如何安装Linux,安装Linux后我们如果只在终端界面下配置会很麻烦,我们可以通过windows连接linux的ssh进行配置linux.SSH 是目前较可靠,专为远程登录会话和其他网络服务提供安全性的协 …</p><p>上一篇介绍了如何安装Linux,安装Linux后我们如果只在终端界面下配置会很麻烦,我们可以通过windows连接linux的ssh进行配置linux.SSH 是目前较可靠,专为远程登录会话和其他网络服务提供安全性的协议。</p>
<p>要想连接ssh,首先就需要保证windows能和虚拟机Linux进行通信,我们如果想要本机能和虚拟机通信需要设置虚拟机的网卡连接关系,有3中连接关系可以和本机连通:
一种是Bridged(桥接),就是使用你本机的连接,如果本地用的是局域网DHCP上网可以选择这种方式.
一种是NAT,就是用NAT模式,就是让虚拟系统借助NAT(网络地址转换)功能,通过宿主机器所在的网络来访问公网。如果本地用的是静态公网ip,本地有无DHCP这个是首选
一种host-only:这种模式不能访问外网,只能何宿主(也就是本机)主机通信.也就不能访问Internet</p>
<p>这里介绍如何设置NAT模式上网.按说网卡选择NAT模式直接重启网卡就可以dhcp获取,但是我们在以后配置Linux服务器的时候为了方便管理肯定不会使用DHCP,所以我们使用静态ip的方式来设置NAT上网.
首先右键你的虚拟机标签->Setting->选中Network Adapter.在右边选中NAT(也可直接双击右下角的网卡图标)
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310101223.jpg"></p>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120310101241.jpg"></p>
<p>做完这步之后我们还要编辑一下虚拟机的虚拟卡设置,以达到我们使用静态ip上网的需求:</p>
<p>点击虚拟机的Edit(编辑)->Virtual Network Editor(虚拟网卡编辑器):
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310101559.jpg">
在弹出的界面选中VMnet8 NAT,然后修改最下面的Subnet IP,改成你想要的网段,这里使用<code>192.168.3.0/24</code>.
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310101834.jpg"></p>
<p>然后点击NAT Settting:设置Gateway IP为<code>192.168.3.0</code>网段的一个ip,这里使用<code>192.168.3.254</code>.
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310102222.jpg"></p>
<p>设置好后然后点击OK保存退出.</p>
<p>一切已经就绪,下面我们就开始我们的Linux之旅.首先进入系统,在login:界面输入root用户,然后输入安装时输入的root密码.进入系统.首先我们来配置网卡.CentOS的网卡路径在"/etc/sysconifg/network-scripts/"下面,第一个网卡一般命名为eth0,网卡配置文件则是ifcfg-eth0,第二个是eth1,配置文件ifcfg-eth1,后面的以此类推.下面我们来配置网卡:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/sysconfig/network-scripts/ifcfg-eth0
</code></pre></div>
<p>(在这里说一下,Linux有一个很方便的功能当路径太长不方便记忆时,可打出一部分敲Tab键如果这个是唯一的就会自动补全,如果不是就敲两下Tab,就会列出所有可选内容,文件和命令一样)
按i进入输入模式,将配置文件改成如下内容:</p>
<div class="highlight"><pre><span></span><code><span class="nv">DEVICE</span><span class="o">=</span>eth0<span class="w"> </span><span class="c1"># 设备名</span>
<span class="nv">HWADDR</span><span class="o">=</span><span class="m">00</span>:00:00:00:01<span class="w"> </span><span class="c1"># MAC地址</span>
<span class="nv">BOOTPROTO</span><span class="o">=</span>static<span class="w"> </span><span class="c1"># IP获取方式为static,如果是dhcp则是使用DHCP获取</span>
<span class="nv">ONBOOT</span><span class="o">=</span>yes<span class="w"> </span><span class="c1"># 是否启用网卡(也就是是否在重启设备或重启网卡时启动这个网卡)</span>
<span class="nv">IPADDR</span><span class="o">=</span><span class="m">192</span>.168.3.120<span class="w"> </span><span class="c1"># IP地址</span>
<span class="nv">NETMASK</span><span class="o">=</span><span class="m">255</span>.255.255.0<span class="w"> </span><span class="c1"># 掩码</span>
<span class="nv">GATEWAY</span><span class="o">=</span><span class="m">192</span>.168.3.254<span class="w"> </span><span class="c1"># 网关(也就是刚才虚拟网卡编辑器里NAT Setting里设置的网关)</span>
</code></pre></div>
<p>编辑完成后按Esc进入末行模式然后输入:wq保存退出
然后重启网卡,如果都是[OK]的就没有问题</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>network<span class="w"> </span>restart
</code></pre></div>
<p>查看网络配置的命令是</p>
<div class="highlight"><pre><span></span><code>ifconfig
</code></pre></div>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120310104548.jpg"></p>
<p>ping命令用来检测网络连通性,配置好后我们ping网关测试:</p>
<div class="highlight"><pre><span></span><code>ping<span class="w"> </span><span class="m">192</span>.168.3.254
</code></pre></div>
<p>如果显示如下则表示是通的,如果不是则表示不通
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310104753.jpg">
如果能正常访问网络我们还需要配置一个DNS服务器,指定一个DNS服务器的配置文件为:/etc/resolv.conf.Linux可以配置3个DNS.</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/resolv.conf
</code></pre></div>
<p>同样按i或a进入输入模式添加如下内容:</p>
<div class="highlight"><pre><span></span><code>nameserver<span class="w"> </span><span class="m">192</span>.168.3.254<span class="w"> </span><span class="c1"># 本地DNS</span>
nameserver<span class="w"> </span><span class="m">202</span>.106.0.20<span class="w"> </span><span class="c1"># 北京地区网通DNS</span>
nameserver<span class="w"> </span><span class="m">8</span>.8.8.8<span class="w"> </span><span class="c1"># google的根DNS</span>
</code></pre></div>
<p>然后按Esc,输入:wq保存退出,然后ping <a href="http://www.linuxzen.com">www.linuxzen.com</a>看能否解析:
<img alt="www.linuxzen.com" src="/upload/未命名.jpg">
上图表示解析成功.</p>
<p>好了到这一步Linux已经可以访问internet了.下面我们就配置ssh连接虚拟机.连接ssh的软件有几款都是比较不错的,个人觉得最好的还是Secure CRT,这个是收费的但较低版本还是可以在百度/谷歌上找到特别码(你懂得),还有一款是putty,这一个款开源软件而且只有几百KB,很轻量级.</p>
<p>我们首先要关闭selinux(一款美国国家安全局贡献出来的软件,如果会配置可以开启,如果不会就关闭)
编辑/etc/selinux/config:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/selinux/config
</code></pre></div>
<p>然后按i进入输入模式,修改SELINUX参数为disabled:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># This file controls the state of SELinux on the system.</span>
<span class="c1"># SELINUX= can take one of these three values:</span>
<span class="c1"># enforcing - SELinux security policy is enforced.</span>
<span class="c1"># permissive - SELinux prints warnings instead of enforcing.</span>
<span class="c1"># disabled - SELinux is fully disabled.</span>
<span class="nv">SELINUX</span><span class="o">=</span>disabled<span class="w"> </span><span class="c1"># 修改此行</span>
<span class="c1"># SELINUXTYPE= type of policy in use. Possible values are:</span>
<span class="c1"># targeted - Only targeted network daemons are protected.</span>
<span class="c1"># strict - Full SELinux protection.</span>
<span class="nv">SELINUXTYPE</span><span class="o">=</span>disabled
</code></pre></div>
<p>然后重启系统</p>
<div class="highlight"><pre><span></span><code>reboot
</code></pre></div>
<p>然后清空系统自带防火墙</p>
<div class="highlight"><pre><span></span><code>iptables<span class="w"> </span>-Z
iptables<span class="w"> </span>-F
iptables<span class="w"> </span>-X
service<span class="w"> </span>iptables<span class="w"> </span>save
</code></pre></div>
<p>然后我们就可以使用ssh连接Linux虚拟机
首先确保Linux启动了ssh,ssh占用TCP的22端口.执行命令查看22端口是否开放:</p>
<div class="highlight"><pre><span></span><code>netstat<span class="w"> </span>-antlp<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="m">22</span>
</code></pre></div>
<p>如果出现如下内容则表示ssh已经启动:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310111326.jpg">
如果没有启动尝试运行下面命令:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>sshd<span class="w"> </span>start<span class="w"> </span><span class="c1"># 启动ssh服务</span>
chkconfig<span class="w"> </span>--add<span class="w"> </span>sshd<span class="w"> </span><span class="c1"># 加入开机启动</span>
</code></pre></div>
<p>如果没报错则sshd启动成功</p>
<p>ssh也配置成功了,下面就需要本地连接ssh了,首先我们要配置本地能访问虚拟机,不知道大家注意到们装完虚拟机我们多出两个链接分别是VMware Network Adapter VMnet1和VMware Network Adapter VMnet8,VMnet1是host only,VMnet8是NAT,我们使用的是NAT,所以配置VMware Network Adapter VMnet8就可以了,把VMware Network Adapter VMnet8配置成192.168.3.x的ip,这里使用192.168.3.111,配置过程这里不做解释了.在windows的cmd下ping 192.168.3.120,能ping通就打开你的ssh客户端软件.这里使用SecureCRT.打开SecureCRT点击快速连接:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310112142.jpg">
在弹出的界面输入主机名处输入ip地址:192.168.3.120,用户名处输入root:然后点击连接
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310112351.jpg"></p>
<p>会弹出是否保存密钥:点击接收并保存:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310112605.jpg"></p>
<p>然后输入root密码既可以连接登录
<img alt="www.linuxzen.com" src="/upload/QQ截图20120310112713.jpg"></p>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120310112752.jpg">
好了到这里就可以用ssh连接Linux了,尽情探索把,下一节将会讲解yum库的配置和包安装.</p>lnmp环境搭建完全手册(一)------系统安装2012-03-08T12:05:00+08:002012-03-08T12:05:00+08:00coldtag:www.linuxzen.com,2012-03-08:/lnmphuan-jing-da-jian-wan-quan-shou-ce-yi-xi-tong-an-zhuang.html<p>在之前的文章里也陆陆续续的介绍了nginx/mysql和php的安装和配置,这里做一个整合.也是一个详细的教程,可以让你0基础搭建lnmp环境.
首先选择一个发行版,个人 …</p><p>在之前的文章里也陆陆续续的介绍了nginx/mysql和php的安装和配置,这里做一个整合.也是一个详细的教程,可以让你0基础搭建lnmp环境.
首先选择一个发行版,个人比较喜欢CentOS,CentOS是Red Hat的再发行版,重新编译了Red Hat,修复n多错误,免费的yum库,这里使用CentOS5.5,虚拟机使用VMware Workstation 8.0.VMware是一个“虚拟PC”软件公司.它可以使你在一台机器上同时运行二个或更多Windows、DOS、LINUX系统。与“多启动”系统相比,VMWare采用了完全不同的概念。多启动系统在一个时刻只能运行一个系统,在系统切换时需要重新启动机器。VMWare是真正“同时”运行,多个操作系统在主系统的平台上,就象标准Windows应用程序那样切换.</p>
<p>安装VMware这里就不介绍了,给出VMware Workstation 8.0的<a href="http://www.vmware.com/downloads/downloadBinary.do?downloadGroup=WKST-802-WIN&vmware=downloadBinary&file=VMware-workstation-full-8.0.2-591240.exe&pot=1&code=VMware-workstation-full-8.0.2-591240.exe&hashKey=5d70214cc3ce52f8154f38b7cd52efbe&tranId=70310191&downloadURL"">下载地址</a></p>
<p>VMware不是免费软件,但是网上会有序列号,大家不想花钱就搜一下就ok了.
我们还要准备CentOS5.5 的iso镜像用来安装系统.<a href=""http://download.chinaunix.net/down.php?id=31679&ResourceID=12271&site=6">下载地址</a></p>
<p>下面我们就来新建一个虚拟机安装CentOS
打开虚拟机选择File->New Virtual Machine
<img alt="www.linuxzen.com" src="/upload/1.jpg"></p>
<p>在接下来的界面选择<code>Typical</code>,然后点<code>Next</code>:
<img alt="www.linuxzen.com" src="/upload/2.jpg"></p>
<p>然后点选<code>I will install the operating system later</code>,然后Next>,选择操作系统,为Linux,Version选择为CentOS:
<img alt="www.linuxzen.com" src="/upload/3.jpg"></p>
<p>点击Next进给虚拟机命名,然后选择保存路径(最好选择一个空闲空间足够的分区)点击Next会让你选择硬盘大小,保持默认即可,点击Next>在弹出的界面可以查看虚拟机的虚拟硬件可以点击"Customize Hardware"进行自定义硬件,建议删除软驱,USB等无用硬件.点击Finish完成虚拟机创建.</p>
<p>创建完成后虚拟机界面的左边双击"CD/DVD(IDE)":
<img alt="www.linuxzen.com" src="/upload/4.jpg">
在弹出界面的右边选择<code>Use ISO image file</code>,点击"Browse.."选择下载好的CentOS5.5镜像,点击OK确定:</p>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120308101907.jpg"></p>
<p>然后单击虚拟机界面的左上角的<code>Power on this virtual machine</code>启动虚拟机,第一次默认会认到光驱从光驱启动,启动后看到如下界面:</p>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120308102129.jpg"></p>
<p>恭喜你,可以开始你的linux之旅了,我比较喜欢字符界面安装,节省资源,输入<code>linux text</code>,敲回车进入字符界面安装,想图形化安装直接回车即可:
在接下来的界面下会弹出下面对话框,按Tab选择Skip跳过检测:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308102415.jpg">
然后会一次弹出语言,键盘选择框选择默认既可以,然后linux会再你的硬盘上检测不到分区表询问你是否格式化整个硬盘,选择Yes:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308103037.jpg">
然后就会跳到分区这一块,我们选择第四个自定义分区(Create Custom layout):
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308103159.jpg"></p>
<p>接下来按F2即可建立分区,一般分区有各种各样的建议,Linux必须有一个/(根)分区,建议有一个swap分区是物理内存的两倍.</p>
<p><img alt="www.linuxzen.com" src="/upload/QQ截图20120308103443.jpg"></p>
<p>上图建立一个/大小为<code>4G</code>,<code>Mount Point</code>指定挂载点,这里为<code>/</code>,<code>File System Type</code>指定文件系统类型,这里为<code>ext3</code>.大小指定为<code>4G</code>.</p>
<p>然后建议一个<code>swap</code>分区,同样按<code>F2</code>在<code>File System Type</code>里选择为<code>swap</code>:
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308103741.jpg">
在根据第一步的方法创建一个/usr分区,在Size里选择"Fill all available space"把剩余的大小都给/usr
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308103957.jpg">
创建完毕选择ok完成分区,跳出是否安装引导(Boot loader Configuration),一直选择默认继续.接下来询问是否配置网卡,选择No跳过.会让你填Gateway和DNS不用管直接ok就行,接下来会让输入主机名,输入你想要的主机名即可
<img alt="www.linuxzen.com" src="/upload/QQ截图20120308104357.jpg"></p>
<p>接下来选择时区,选择Asia/Chongqing.然后输入root密码,输入完毕后OK继续,下面就是选择要安装的包,想玩桌面的话就直接默认,这里选择Customize software selection,接下来的界面按空格去掉所有的包前面的*实现最小化安装,选择OK开始安装.完成后重启,linux系统就安装完成</p>nginx平台搭建nagios监控系统2012-02-26T10:48:00+08:002012-02-26T10:48:00+08:00coldtag:www.linuxzen.com,2012-02-26:/nginxping-tai-da-jian-nagiosjian-kong-xi-tong.html<p>Nagios是一款开源的免费网络监视工具,能有效监控Windows、Linux和Unix的主机状态,交换机路由器等网络设置,打印机等。在系统或服务状态异常时发出邮件或短信报警第一时间通知网 …</p><p>Nagios是一款开源的免费网络监视工具,能有效监控Windows、Linux和Unix的主机状态,交换机路由器等网络设置,打印机等。在系统或服务状态异常时发出邮件或短信报警第一时间通知网站运维人员,在状态恢复后发出正常的邮件或短信通知。</p>
<p>Nagios 功能:</p>
<ul>
<li>监视网络服务 (SMTP, POP3, HTTP, NNTP, PING等)</li>
<li>监视主机资源 (进程, 磁盘等)</li>
<li>简单的插件设计可以轻松扩展Nagios的监视功能</li>
<li>服务等监视的并发处理</li>
<li>错误通知功能 (通过email, pager, 或其他用户自定义方法)</li>
<li>可指定自定义的事件处理控制器</li>
<li>可选的基于浏览器的WEB界面以方便系统管理人员查看网络状态,各种系统问题,以及日志等等</li>
<li>可以通过手机查看系统监控信息</li>
</ul>
<p>本文环境:</p>
<ul>
<li>系统:CentOS 5.5 32bit</li>
<li>ip:192.168.3.3</li>
<li>nagios core:nagios-3.3.1.tar.gz</li>
<li>web:nginx(nginx-1.0.5.tar.gz)</li>
</ul>
<h2 id="_1">安装:</h2>
<h3 id="_2">安装准备</h3>
<p>创建nagios系统用户:</p>
<div class="highlight"><pre><span></span><code>useradd<span class="w"> </span>nagios
</code></pre></div>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>gd-devel<span class="w"> </span>libpng-devel<span class="w"> </span>libtool-ltdl-devel
</code></pre></div>
<p>下载nagios内核:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.3.1.tar.gz
</code></pre></div>
<h3 id="nagios">安装nagios内核:</h3>
<div class="highlight"><pre><span></span><code>tar<span class="w"> </span>-zxvf<span class="w"> </span>nagios-3.3.1.tar.gz
<span class="nb">cd</span><span class="w"> </span>nagios
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nagios
make<span class="w"> </span>all
make<span class="w"> </span>install<span class="w"> </span><span class="c1"># 安装主要程序,CGI和HTML</span>
make<span class="w"> </span>install-commandmode<span class="w"> </span><span class="c1"># 使外部命令有访问nagios配置文件的权限</span>
make<span class="w"> </span>install-config<span class="w"> </span><span class="c1"># 安装配置文件</span>
make<span class="w"> </span>install-init<span class="w"> </span><span class="c1"># 安装nagios启动脚本</span>
</code></pre></div>
<h3 id="_3">安装插件:</h3>
<p>nagios没有什么都做不了的,插件极大的扩展了nagios功能,除了安装官方常用插件外,也可以根据自己需求编写插件.</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>nagios-plugins-1.4.15.tar.gz
<span class="nb">cd</span><span class="w"> </span>nagios-plugins-1.4.15
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nagios
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<h3 id="web">安装WEB服务器</h3>
<p>web服务器不是nagios所必须的,但是如果你无法忍受通过日志文件来监控各个主机状态,那么web服务器就至关重要,所以我们还是花上一点时间为之后的节省更多的时间.</p>
<p>这里web服务器使用当前主流的nginx:</p>
<p>安装nginx</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://nginx.org/download/nginx-1.0.11.tar.gz
<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>yum<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>-y<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>install<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>zlib-devel<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>pcre-devel<span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span>openssl-devel<span class="w"> </span><span class="p">&</span>amp<span class="p">;</span>nbsp<span class="p">;</span><span class="c1"># 安装依赖</span>
tar<span class="w"> </span>-zxvf<span class="w"> </span>nginx-1.0.11.tar.gz
<span class="nb">cd</span><span class="w"> </span>nginx-1.0.11
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nginx<span class="w"> </span>--with-http_ssl_module<span class="w"> </span>--with-http_flv_module<span class="w"> </span>--with-http_gzip_static_module
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>配置nginx支持cgi:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://www.cpan.org/modules/by-module/FCGI/FCGI-0.67.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>FCGI-0.67.tar.gz
<span class="nb">cd</span><span class="w"> </span>FCGI-0.67
perl<span class="w"> </span>Makefile.PL
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
<span class="nb">cd</span><span class="w"> </span>..
wget<span class="w"> </span>http://cpan.wenzk.com/authors/id/G/GB/GBJK/FCGI-ProcManager-0.19.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>FCGI-ProcManager-0.19.tar.gz
<span class="nb">cd</span><span class="w"> </span>FCGI-ProcManager-0.19
perl<span class="w"> </span>Makefile.PL
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
<span class="nb">cd</span><span class="w"> </span>..
wget<span class="w"> </span>http://search.cpan.org/CPAN/authors/id/I/IN/INGY/IO-All-0.39.tar.gz
tar<span class="w"> </span>zxvf<span class="w"> </span>IO-All-0.39.tar.gz
<span class="nb">cd</span><span class="w"> </span>IO-All-0.39
perl<span class="w"> </span>Makefile.PL
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>创建cgiwarp-fcgi.pl(内容参照<a href="http://wiki.nginx.org/NginxSimpleCGI">这里</a>),将cgiwarp-fcgi.pl放到/usr/local/bin下给执行权限,并执行:</p>
<div class="highlight"><pre><span></span><code>chmod<span class="w"> </span>+x<span class="w"> </span>/usr/local/bin/cgiwarp-fcgi.pl
mkdir<span class="w"> </span>-p<span class="w"> </span>/var/run/nginx/
perl<span class="w"> </span>/usr/local/bin/cgiwarp-fcgi.pl
chown<span class="w"> </span>nobody.nobody<span class="w"> </span>/var/run/nginx/cgiwrap-dispatch.sock<span class="w"> </span><span class="c1"># 改变属主为nobody,是nginx正常使用</span>
</code></pre></div>
<p>配置nginx支持身份验证</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>httpd
chkconfig<span class="w"> </span>--del<span class="w"> </span>httpd
mkdir<span class="w"> </span>/usr/local/nginx/conf/htpasswd
htpasswd<span class="w"> </span>-c<span class="w"> </span>/usr/local/nginx/conf/htpasswd/nagios<span class="w"> </span>nagiosadmin<span class="w"> </span><span class="c1"># 添加nagiosadmin用户</span>
</code></pre></div>
<p>还需要配置nginx支持php,这里使用php-fpm的方式,这里就不介绍.</p>
<h2 id="_4">配置</h2>
<p>首先配置nginx:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.3:80<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span><span class="m">192</span>.168.3.3<span class="p">;</span>
<span class="w"> </span>auth_basic<span class="w"> </span><span class="s2">"Nagios"</span><span class="p">;</span>
<span class="w"> </span>auth_basic_user_file<span class="w"> </span>/usr/local/nginx/conf/htpasswd/nagios<span class="p">;</span>
<span class="w"> </span><span class="c1"># 为了正常使用css和显示图片做一个url重写</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">(</span><span class="w"> </span><span class="nv">$request_filename</span><span class="w"> </span>~<span class="w"> </span><span class="se">\.</span><span class="o">(</span>gif¦png¦jpg¦jpeg¦ico<span class="o">)</span><span class="w"> </span><span class="o">)</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>rewrite<span class="w"> </span>^/nagios/<span class="o">(</span>images/.*<span class="o">)</span><span class="w"> </span>/<span class="nv">$1</span><span class="w"> </span>break<span class="p">;</span>
<span class="w"> </span><span class="o">}</span><span class="w"> </span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">(</span><span class="w"> </span><span class="nv">$request_filename</span><span class="w"> </span>~<span class="w"> </span><span class="se">\.</span><span class="o">(</span>css<span class="o">)</span><span class="w"> </span><span class="o">)</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>rewrite<span class="w"> </span>^/nagios/<span class="o">(</span>stylesheets/.*<span class="o">)</span><span class="w"> </span>/<span class="nv">$1</span><span class="w"> </span>break<span class="p">;</span>
<span class="w"> </span><span class="o">}</span><span class="w"> </span>
<span class="w"> </span><span class="c1"># 支持php的配置</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>/usr/local/nagios/share/<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.php<span class="p">;</span>
<span class="w"> </span>fastcgi_pass<span class="w"> </span>unix:/var/run/php/php-fpm.sock<span class="p">;</span>
<span class="w"> </span>fastcgi_index<span class="w"> </span>index.php<span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SCRIPT_FILENAME<span class="w"> </span><span class="nv">$document_root</span>/<span class="nv">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span>include<span class="w"> </span>fastcgi_params<span class="p">;</span>
<span class="w"> </span>include<span class="w"> </span>fastcgi.conf<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="c1"># 支持cgi的配置</span>
<span class="w"> </span>location<span class="w"> </span>~<span class="w"> </span><span class="se">\.</span>cgi$<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>/usr/local/nagios/sbin/<span class="p">;</span>
<span class="w"> </span>rewrite<span class="w"> </span>^/nagios/cgi-bin/<span class="o">(</span>.*<span class="o">)</span><span class="se">\.</span>cgi<span class="o">(</span>.*<span class="o">)</span><span class="w"> </span>/<span class="nv">$1</span>.cgi<span class="nv">$2</span><span class="w"> </span>break<span class="p">;</span>
<span class="w"> </span>fastcgi_pass<span class="w"> </span>unix:/var/run/nginx/cgiwrap-dispatch.sock<span class="p">;</span>
<span class="w"> </span>fastcgi_index<span class="w"> </span>index.cgi<span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SCRIPT_FILENAME<span class="w"> </span><span class="nv">$document_root</span>/<span class="nv">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>QUERY_STRING<span class="w"> </span><span class="nv">$query_string</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>REQUEST_METHOD<span class="w"> </span><span class="nv">$request_method</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>CONTENT_TYPE<span class="w"> </span><span class="nv">$content_type</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>CONTENT_LENGTH<span class="w"> </span><span class="nv">$content_length</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>GATEWAY_INTERFACE<span class="w"> </span>CGI/1.1<span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SERVER_SOFTWARE<span class="w"> </span>nginx<span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SCRIPT_NAME<span class="w"> </span><span class="nv">$fastcgi_script_name</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>REQUEST_URI<span class="w"> </span><span class="nv">$request_uri</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>DOCUMENT_URI<span class="w"> </span><span class="nv">$document_uri</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>DOCUMENT_ROOT<span class="w"> </span><span class="nv">$document_root</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SERVER_PROTOCOL<span class="w"> </span><span class="nv">$server_protocol</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>REMOTE_ADDR<span class="w"> </span><span class="nv">$remote_addr</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>REMOTE_PORT<span class="w"> </span><span class="nv">$remote_port</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SERVER_ADDR<span class="w"> </span><span class="nv">$server_addr</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SERVER_PORT<span class="w"> </span><span class="nv">$server_port</span><span class="p">;</span>
<span class="w"> </span>fastcgi_param<span class="w"> </span>SERVER_NAME<span class="w"> </span><span class="nv">$server_name</span><span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
</code></pre></div>
<p>修改nagios主配文件,修改nagios运行账户:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/usr/local/nagios/etc/nagios.cfg
<span class="c1"># 编辑下面选项:</span>
<span class="nv">nagios_user</span><span class="o">=</span>nobody
</code></pre></div>
<p>修改cgi.cfg配置文件
修改默认认证cgi用户的为刚才用htpasswd创建的nagiosadmin,或用htpasswd新建一个用户用来认证cgi文件(不然nagios报错:<code>It appears as though you do not have permission to view information for any of the hosts you requested...</code>)</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/usr/local/nagios/etc/cgi.cfg
<span class="nv">default_user_name</span><span class="o">=</span>nagiosadmin
</code></pre></div>
<p>修改localhost.cfg添加一个新的监控主机:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/usr/local/nagios/etc/objects/localhost.cfg
<span class="c1"># 通过定义host添加一个主机</span>
<span class="c1"># 添加一个ip为192.168.3.102的监控主机</span>
define<span class="w"> </span>host<span class="o">{</span>
<span class="w"> </span>use<span class="w"> </span>linux-server
<span class="w"> </span>host_name<span class="w"> </span>slave.org
<span class="w"> </span><span class="nb">alias</span><span class="w"> </span><span class="m">102</span>
<span class="w"> </span>address<span class="w"> </span><span class="m">192</span>.168.3.102
<span class="w"> </span><span class="o">}</span>
....
<span class="c1"># 通过定义service定义监控的服务:</span>
<span class="c1"># 通过ping来监控主机是否存活</span>
define<span class="w"> </span>service<span class="o">{</span>
<span class="w"> </span>use<span class="w"> </span>local-service
<span class="w"> </span>host_name<span class="w"> </span>slave.org
<span class="w"> </span>service_description<span class="w"> </span>PING
<span class="w"> </span>check_command<span class="w"> </span>check_ping!100.0,20%!500.0,%60
<span class="w"> </span><span class="o">}</span>
</code></pre></div>
<p>修改nagios安装目录权限:</p>
<div class="highlight"><pre><span></span><code>chown<span class="w"> </span>-R<span class="w"> </span>nobody.nagios<span class="w"> </span>/usr/local/nagios
chmod<span class="w"> </span>-R<span class="w"> </span><span class="m">775</span><span class="w"> </span>/usr/local/nagios/
</code></pre></div>
<p>检查nagios配置文件:</p>
<div class="highlight"><pre><span></span><code>/usr/local/nagios/bin/nagios<span class="w"> </span>-v<span class="w"> </span>/usr/local/nagios/etc/nagios.cfg
</code></pre></div>
<p>在最后看到如下则表示成功:</p>
<p>Total Warnings: 0
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check</p>
<p>启动nagios:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>nagios<span class="w"> </span>start
Starting<span class="w"> </span>nagios:<span class="w"> </span><span class="k">done</span>.
</code></pre></div>
<p>这时候在浏览器里输入<code>http://ip/</code>查看状态,现在能看到所有监控主机的的状态了.</p>
<p><img alt="www.linuxzen.com" src="/upload/blog001.jpg"></p>搭建ntop监测局域网内流量2012-02-20T17:37:00+08:002012-02-20T17:37:00+08:00coldtag:www.linuxzen.com,2012-02-20:/da-jian-ntopjian-ce-ju-yu-wang-nei-liu-liang.html<p>ntop是一种监控网络流量的工具,用ntop显示网络的使用情况比其他一些网管软件更加直观、详细。ntop甚至可以列出每个节点计算机的网络带宽利用率。ntop是一个灵活的、功能齐全的,用来监控和解决局域网问题的工具。它同时提供命令行输入和Web界面,可应用于嵌入式Web服务。</p>
<p>本文环 …</p><p>ntop是一种监控网络流量的工具,用ntop显示网络的使用情况比其他一些网管软件更加直观、详细。ntop甚至可以列出每个节点计算机的网络带宽利用率。ntop是一个灵活的、功能齐全的,用来监控和解决局域网问题的工具。它同时提供命令行输入和Web界面,可应用于嵌入式Web服务。</p>
<p>本文环境:CentOS5.5 32位</p>
<ul>
<li>ip:192.168.3.101</li>
<li>rrdtool:rrdtool-1.4.7.tar.gz</li>
<li>GeoIP:GeoIP-1.4.8.tar.gz</li>
<li>ntop:ntop-4.1.0.tar.gz</li>
</ul>
<h2 id="_1">一.安装依赖:</h2>
<h3 id="rrdtool">安装rrdtool</h3>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>libxml2<span class="w"> </span>glib-devel<span class="w"> </span>pango-devel
wget<span class="w"> </span>http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.7.tar.gz<span class="w"> </span><span class="c1"># 下载</span>
tar<span class="w"> </span>-zxvf<span class="w"> </span>rrdtool-1.4.7.tar.gz
<span class="nb">cd</span><span class="w"> </span>rrdtool-1.4.7
./configure<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<h3 id="geoip">安装geoip:</h3>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://geolite.maxmind.com/download/geoip/api/c/GeoIP-1.4.8.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>GeoIP-1.4.8.tar.gz
<span class="nb">cd</span><span class="w"> </span>GeoIP-1.4.8
./configure<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<h3 id="_2">安装其余依赖:</h3>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>libtool<span class="w"> </span>libpcap-devel<span class="w"> </span>gd-devel<span class="w"> </span>gdbm-devel<span class="w"> </span>openssl-devel
<span class="w"> </span>intltool
</code></pre></div>
<p> </p>
<h2 id="_3">二.下载安装:</h2>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://sourceforge.net/projects/ntop/files/ntop/Stable/ntop-4.1.0.tar.gz/download
tar<span class="w"> </span>-zxvf<span class="w"> </span>ntop-4.1.0.tar.gz
<span class="nb">cd</span><span class="w"> </span>ntop-4.1.0
./autogen.sh<span class="w"> </span>--with-tcpwrap<span class="w"> </span>--with-rrd-home<span class="o">=</span>/opt/rrdtool-1.4.7/
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<h2 id="_4">三.配置启动</h2>
<p>更改检测数据保存目录的权限:</p>
<div class="highlight"><pre><span></span><code>chown<span class="w"> </span>-R<span class="w"> </span>nobody<span class="w"> </span>/usr/local/var/ntop/
</code></pre></div>
<p>创建admin管理密码:</p>
<div class="highlight"><pre><span></span><code>ntop<span class="w"> </span>-A
</code></pre></div>
<p>在eth0上启动ntop:</p>
<div class="highlight"><pre><span></span><code>ntop<span class="w"> </span>-d<span class="w"> </span>-i<span class="w"> </span>eth0<span class="w"> </span>-M
</code></pre></div>
<p>客户端浏览监测结果:
在浏览器上输入:http://192.168.3.101:3000访问即可查看监测结果,Admin可以配置ntop.到此这个监控局域网流量的ntop就搭建完成,这样就可以很方便的查看局域网流量,可以发现是否有人在执行arp攻击等等常见攻击.剩余的功能大家就自己发现吧,这里不错介绍.</p>
<p>如果英语阅读困难的话可以,有牛人已经做出来了中文版,大家可以下载安装.</p>Linux一些比较实用的小技巧2012-02-20T11:30:00+08:002012-02-20T11:30:00+08:00coldtag:www.linuxzen.com,2012-02-20:/linuxyi-xie-bi-jiao-shi-yong-de-xiao-ji-qiao.html<p>在Linux中各种各样的小技巧可以帮助我们更好更快的完成我们的工作,下面就介绍一些我所知道的小技巧</p>
<h2 id="_1">文件查找</h2>
<p>找出最近修改的文件:</p>
<div class="highlight"><pre><span></span><code>find<span class="w"> </span>/<span class="w"> </span>-ctime<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 找出根下最近24小时修改过inode信息的文件(更改权限)</span>
find<span class="w"> </span>/<span class="w"> </span>-mtime<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 找出根 …</span></code></pre></div><p>在Linux中各种各样的小技巧可以帮助我们更好更快的完成我们的工作,下面就介绍一些我所知道的小技巧</p>
<h2 id="_1">文件查找</h2>
<p>找出最近修改的文件:</p>
<div class="highlight"><pre><span></span><code>find<span class="w"> </span>/<span class="w"> </span>-ctime<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 找出根下最近24小时修改过inode信息的文件(更改权限)</span>
find<span class="w"> </span>/<span class="w"> </span>-mtime<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 找出根下最近24小时修改过的文件(内容)</span>
find<span class="w"> </span>/<span class="w"> </span>-atime<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 找出根下最近24小时访问过的文件</span>
</code></pre></div>
<p>使用通配符查找文件</p>
<div class="highlight"><pre><span></span><code>find<span class="w"> </span>/<span class="w"> </span>-name<span class="w"> </span><span class="s2">"*.log"</span><span class="w"> </span><span class="c1"># 找出根下以log为后缀的文件,这里必须要加双引号,不然会报错,因为找的是多个文件,需要用双引号引起来</span>
</code></pre></div>
<h2 id="_2">文本替换</h2>
<div class="highlight"><pre><span></span><code>sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'2s/ext3/ext4/'</span><span class="w"> </span>/etc/fstab<span class="w"> </span><span class="c1"># 将第二行的ext3改成ext4</span>
</code></pre></div>
<h2 id="_3">显示指定行</h2>
<div class="highlight"><pre><span></span><code>sed<span class="w"> </span>-n<span class="w"> </span><span class="s2">"3p"</span><span class="w"> </span>/etc/fstab<span class="w"> </span><span class="c1"># 显示第3行</span>
sed<span class="w"> </span>-n<span class="w"> </span><span class="s2">"3,5p"</span><span class="w"> </span>/etc/gfstab<span class="w"> </span><span class="c1"># 显示第3到5行</span>
sed<span class="w"> </span>-n<span class="w"> </span><span class="s2">"3p;5p"</span><span class="w"> </span>/etc/fstab<span class="w"> </span><span class="c1"># 显示第3行和第5行</span>
awk<span class="w"> </span><span class="s1">'NR==3'</span><span class="w"> </span>/etc/fstab<span class="w"> </span><span class="c1"># 显示第3行</span>
</code></pre></div>
<h2 id="_4">在行首或行尾添加:</h2>
<div class="highlight"><pre><span></span><code>sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'s/^/hello'</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span><span class="c1"># 在test文件的行首添加hello</span>
sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'s/$/hello'</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span><span class="c1"># 在test文件的行尾添加hello</span>
sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">'3s/^/hello'</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span><span class="c1"># 在test文件的第3行行首添加hello</span>
</code></pre></div>
<h2 id="vi">在vi里执行命令:</h2>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/sysconfig/networ-scripts/ifcfg-eth0
<span class="c1"># 打开vi,在末行模式下(ESC-&gt;:)</span>
:r!cat<span class="w"> </span>/mnt/ip.txt<span class="w"> </span><span class="c1"># 在当前行的下面输入ip.txt的内容,r代表命令输出放到下一行,!后面是要执行的命令</span>
:.!cat<span class="w"> </span>/mnt/ip.txt<span class="w"> </span><span class="c1"># 在当前行输入ip.txt的内容,.代表将命令输出放到当前航</span>
</code></pre></div>
<h2 id="_5">执行上一条命令:</h2>
<p>如果刚执行了一条server network restart,如果又做了一些操作,需要再次执行,按上键调处可以,还有更快捷的就是:</p>
<div class="highlight"><pre><span></span><code>!ser
</code></pre></div>
<p>!会在命令历史找与!后面匹配的最近一条命令.</p>
<h2 id="ssh">通过ssh连接服务器上传下载文件:</h2>
<p>当你通过ssh连接你的Linux的时候你想下载一个文件到本地,或者像把本地文件上传到远端,你想到用ftp吗?out了,当你通过ssh连接的时候Linux提供了两个命令 rz/sz,命令依赖于包lrzsz.rz命令可以将本地文件通过ssh上传到Linux上,sz pkg 可以将远端的pkg下载到本地.</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>lrzsz<span class="w"> </span><span class="c1"># 安装</span>
</code></pre></div>
<p>然后就可以通过rz/sz来上传下载文件,省去了文件服务器,也可更方便更快捷的管理你的服务器</p>
<h2 id="windowslinux">windows文件转换为Linux格式的文件</h2>
<p>windows的文件格式比Linux格式的文件多了一个回车符\r,可以通过命令来实现转换:
直接转换,dos2unix依赖于包dos2unix:</p>
<div class="highlight"><pre><span></span><code>dos2unix<span class="w"> </span>file.txt<span class="w"> </span><span class="c1"># 需要安装dos2unix包</span>
</code></pre></div>
<p>通过删除\r来实现转换:</p>
<div class="highlight"><pre><span></span><code>tr<span class="w"> </span>-d<span class="w"> </span><span class="s1">'\r'</span><span class="w"> </span>file.txt<span class="w"> </span><span class="p">&</span>gt<span class="p">;&</span>gt<span class="p">;</span><span class="w"> </span>file1.txt<span class="w"> </span><span class="c1"># 通过tr命令删除\r字符,并重定向追加到file1.txt</span>
</code></pre></div>编写Linux shell脚本来实现nginx日志分割2012-02-17T10:27:00+08:002012-02-17T10:27:00+08:00coldtag:www.linuxzen.com,2012-02-17:/bian-xie-linux-shelljiao-ben-lai-shi-xian-nginxri-zhi-fen-ge.html<p>nginx的accss日志每天都会产生大量的日志,不过不进行切割会使查看日志变得异常艰难,这里编写一个脚本结合crond来实现nginx的日志切割,切割的格式为日志后缀的数字越小表示离当 …</p><p>nginx的accss日志每天都会产生大量的日志,不过不进行切割会使查看日志变得异常艰难,这里编写一个脚本结合crond来实现nginx的日志切割,切割的格式为日志后缀的数字越小表示离当前日期越近,比如access.log.2存放的内容要比access.log.1的内容要早.</p>
<p>好了,废话不多说,脚本内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="c1"># Author : cold night</span>
<span class="c1"># Filename : nglogcut.sh</span>
<span class="nb">export</span><span class="w"> </span><span class="nv">PATH</span><span class="o">=</span>/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
<span class="nv">LogPath</span><span class="o">=</span><span class="s1">'/usr/local/nginx/logs/access.log'</span><span class="w"> </span><span class="c1"># 定义日志绝对路径</span>
<span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>ls<span class="w"> </span><span class="nv">$LogPath</span>*<span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span>-F<span class="s1">'/'</span><span class="w"> </span><span class="s1">'{print $NF}'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sort<span class="w"> </span>-nr<span class="sb">`</span>
<span class="k">do</span>
<span class="w"> </span><span class="nv">LastNum</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span><span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span>-F<span class="s1">'.'</span><span class="w"> </span><span class="s1">'{print $NF}'</span><span class="sb">`</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="nv">$LastNum</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>-E<span class="w"> </span><span class="s2">"[0-9]+"</span><span class="w"> </span><span class="m">2</span>>/dev/null<span class="w"> </span><span class="m">1</span>><span class="p">&</span><span class="m">2</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nv">OnNum</span><span class="o">=</span><span class="sb">`</span>expr<span class="w"> </span><span class="nv">$LastNum</span><span class="w"> </span>+<span class="w"> </span><span class="m">1</span><span class="sb">`</span>
<span class="w"> </span>mv<span class="w"> </span><span class="nv">$LogPath</span>.<span class="nv">$LastNum</span><span class="w"> </span><span class="nv">$LogPath</span>.<span class="nv">$OnNum</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span>mv<span class="w"> </span><span class="nv">$LogPath</span><span class="w"> </span><span class="nv">$LogPath</span>.1
<span class="w"> </span><span class="nb">kill</span><span class="w"> </span>-HUP<span class="w"> </span><span class="sb">`</span>cat<span class="w"> </span>/usr/local/nginx/logs/nginx.pid<span class="sb">`</span>
<span class="w"> </span><span class="k">fi</span>
<span class="k">done</span>
</code></pre></div>
<p>然后保存为nglogcut.sh存放在/root/下,下面给脚本赋予执行权限</p>
<div class="highlight"><pre><span></span><code>chmod<span class="w"> </span>+x<span class="w"> </span>/root/nglogcut.sh
</code></pre></div>
<p>配置crond自动执行:</p>
<div class="highlight"><pre><span></span><code>crontab<span class="w"> </span>-e<span class="w"> </span><span class="c1"># 执行crontab -e为当前用户添加,但必须要再脚本前面声明PATH路径,或命令用绝对路径</span>
<span class="c1"># 添加如下内容:</span>
<span class="m">00</span><span class="w"> </span><span class="m">2</span><span class="w"> </span>*/3<span class="w"> </span>*<span class="w"> </span>*<span class="w"> </span>/bin/sh<span class="w"> </span>/root/nglogcut.sh<span class="w"> </span><span class="m">2</span>><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">1</span>><span class="p">&</span><span class="m">2</span><span class="w"> </span><span class="c1"># 每3天执行日志分割(可根据自己情况来定义执行周期)</span>
</code></pre></div>
<hr>
<p><strong>Update 2019-02-20</strong>: 最佳实践应该是使用 <a href="https://linux.die.net/man/8/logrotate">logrotate</a></p>heartbeat实现MySQL双机高可用2012-02-12T17:01:00+08:002012-02-12T17:01:00+08:00coldtag:www.linuxzen.com,2012-02-12:/heartbeatshi-xian-mysqlshuang-ji-gao-ke-yong.html<p>对于一个网站或一个企业最重要的无疑就是数据,那么数据库的数据安全无疑就更加重要,所以我们必须保证数据库的 …</p><p>对于一个网站或一个企业最重要的无疑就是数据,那么数据库的数据安全无疑就更加重要,所以我们必须保证数据库的数据完整,这里就介绍使用heartbeat来实现MySQL双机高可用.</p>
<p>当我们的MySQL数据库故障或MySQL数据库服务器出现故障的时候我们希望有一个备用能自动代替主MySQL数据来完成当前的任务,当主MySQL服务器恢复故障的时候备用的能切换到备用等待下一次故障出现.这里我们就结合故障检测HA来实现.</p>
<p>HA会定时发送心跳包检测主备服务器的健康状态,当主服务器出现故障时会自动将vip切换到备用服务器,由备用服务器执行主服务器的任务,MySQL要实现这样的功能就必须保证主备服务器的数据一致.这就要用到MySQL主从双机.</p>
<!--more-->
<p>本文使用环境:
系统:CentOS 5.5 32位
主MySQL: ip 192.168.3.101/24 主机名:master.org
备用MySQL:192.168.3.102/24 主机名:slave.org
vip:192.168.3.103/24
MySQL:mysql-5.0.95.tar.gz
heartbeat:Heartbeat-3-0-7e3a82377fa8.tar.bz2</p>
<h3 id="mysql">一、安装部署MySQL</h3>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>ncurses-devel<span class="w"> </span>openssl-devel
wget<span class="w"> </span>http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-5.0.95.tar.gz/from/http://mysql.cdpa.nsysu.edu.tw/
useradd<span class="w"> </span>-M<span class="w"> </span>-s<span class="w"> </span>/sbin/nologin<span class="w"> </span>mysql
tar<span class="w"> </span>-zxvf<span class="w"> </span>mysql-5.0.95.tar.gz
<span class="nb">cd</span><span class="w"> </span>mysql-5.0.95
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/mysql<span class="w"> </span><span class="se">\</span>
--without-debug<span class="w"> </span><span class="se">\</span>
--with-extra-charsets<span class="o">=</span>utf8,gbk<span class="w"> </span><span class="se">\</span>
--enable-assembler<span class="w"> </span><span class="se">\</span>
--with-mysqld-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\</span>
--with-client-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\</span>
--with-unix-socket-path<span class="o">=</span>/tmp/mysql.sock<span class="w"> </span><span class="se">\</span>
--with-ssl
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
cp<span class="w"> </span>support-files/my-medium.cnf<span class="w"> </span>/etc/my.cnf<span class="w"> </span><span class="c1"># 创建配置文件</span>
cp<span class="w"> </span>support-files/mysql.server<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span><span class="c1"># 创建启动脚本</span>
chmod<span class="w"> </span>+x<span class="w"> </span>/etc/init.d/mysqld
<span class="nb">echo</span><span class="w"> </span><span class="s1">'/usr/local/mysql/lib/mysql/'</span><span class="w"> </span><span class="p">&</span>gt<span class="p">;&</span>gt<span class="p">;</span><span class="w"> </span>/etc/ld.so.conf
ldconfig
/usr/local/mysql/bin/mysql_install_db<span class="w"> </span>--user<span class="o">=</span>mysql<span class="w"> </span><span class="c1"># 初始化数据库</span>
chown<span class="w"> </span>-R<span class="w"> </span>root.mysql<span class="w"> </span>/usr/local/mysql/
chown<span class="w"> </span>-R<span class="w"> </span>mysql.mysql<span class="w"> </span>/usr/local/mysql/var/
ln<span class="w"> </span>-s<span class="w"> </span>/usr/local/mysql/bin/*<span class="w"> </span>/usr/local/bin/<span class="w"> </span><span class="c1"># 为二进制文件做一个软链接</span>
</code></pre></div>
<p>配置MySQl主从实现数据同步,在主从服务器上修改my.cnf(这里是新安装的数据库,如果是仅仅加从库,需要把主库的数据备份导入到从库,这里不再讲述)</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/my.cnf
<span class="c1"># [mysqld]里修改:</span>
<span class="nv">log_bin</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>/var/log/mysql/mysql-bin.log<span class="w"> </span><span class="c1"># 启动二进制文件</span>
server-id<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1921683101</span><span class="w"> </span><span class="c1"># 设置服务器id</span>
</code></pre></div>
<p>启动主库:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>mysqld<span class="w"> </span>start
</code></pre></div>
<p>在主库上创建一个用户授权给从库,用户为backup密码为backup:</p>
<div class="highlight"><pre><span></span><code>mysql<span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>grant<span class="w"> </span>replication<span class="w"> </span>slave<span class="w"> </span>on<span class="w"> </span>*.*<span class="w"> </span>to<span class="w"> </span><span class="s1">'backup'</span>@<span class="s1">'192.168.3.102'</span><span class="w"> </span>identified<span class="w"> </span>by<span class="w"> </span><span class="s1">'backup'</span><span class="p">;</span>
Query<span class="w"> </span>OK,<span class="w"> </span><span class="m">0</span><span class="w"> </span>rows<span class="w"> </span>affected<span class="w"> </span><span class="o">(</span><span class="m">0</span>.16<span class="w"> </span>sec<span class="o">)</span>
</code></pre></div>
<p>查看主库状态:</p>
<div class="highlight"><pre><span></span><code>mysql<span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>show<span class="w"> </span>master<span class="w"> </span>status<span class="p">;</span>
+------------------+-----------+--------------+------------------+
¦<span class="w"> </span>File<span class="w"> </span>¦<span class="w"> </span>Position<span class="w"> </span>¦<span class="w"> </span>Binlog_Do_DB<span class="w"> </span>¦<span class="w"> </span>Binlog_Ignore_DB<span class="w"> </span>¦
+------------------+-----------+--------------+------------------+
¦<span class="w"> </span>mysql-bin.000003<span class="w"> </span>¦<span class="w"> </span><span class="m">236</span><span class="w"> </span>¦<span class="w"> </span>¦<span class="w"> </span>¦
+------------------+-----------+--------------+------------------+
<span class="m">1</span><span class="w"> </span>row<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="nb">set</span><span class="w"> </span><span class="o">(</span><span class="m">0</span>.00<span class="w"> </span>sec<span class="o">)</span>
</code></pre></div>
<p>修改从库配置文件:</p>
<div class="highlight"><pre><span></span><code>server-id<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1921683102</span><span class="w"> </span><span class="c1"># server id必须保持唯一</span>
<span class="nv">log_bin</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>/var/log/mysql/mysql-bin.log<span class="w"> </span><span class="c1"># 启用二进制日志</span>
master-host<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">192</span>.168.3.101<span class="w"> </span><span class="c1"># 主库ip</span>
master-user<span class="w"> </span><span class="o">=</span><span class="w"> </span>backup<span class="w"> </span><span class="c1"># 账号</span>
master-pass<span class="w"> </span><span class="o">=</span><span class="w"> </span>backup<span class="w"> </span><span class="c1"># 密码</span>
master-port<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3306</span><span class="w"> </span><span class="c1"># 连接主库的端口</span>
master-connect-retry<span class="o">=</span><span class="m">60</span><span class="w"> </span><span class="c1"># 连接失败后进行重试等待的描述</span>
</code></pre></div>
<p>启动从库,并查看状态:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>mysqld<span class="w"> </span>start
</code></pre></div>
<p>在从库上执行下操作,指定主库的二进制文件名和偏移量(刚才在主库show master status;查看的参数):</p>
<div class="highlight"><pre><span></span><code>mysql<span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>show<span class="w"> </span>slave<span class="w"> </span>status<span class="w"> </span><span class="se">\G</span><span class="p">;</span>
***************************<span class="w"> </span><span class="m">1</span>.<span class="w"> </span>row<span class="w"> </span>***************************
<span class="w"> </span>Slave_IO_State:<span class="w"> </span>Waiting<span class="w"> </span><span class="k">for</span><span class="w"> </span>master<span class="w"> </span>to<span class="w"> </span>send<span class="w"> </span>event
<span class="w"> </span>Master_Host:<span class="w"> </span><span class="m">192</span>.168.3.101
<span class="w"> </span>Master_User:<span class="w"> </span>backup
<span class="w"> </span>Master_Port:<span class="w"> </span><span class="m">3306</span>
<span class="w"> </span>Connect_Retry:<span class="w"> </span><span class="m">60</span>
<span class="w"> </span>Master_Log_File:<span class="w"> </span>mysql-bin.000003
<span class="w"> </span>Read_Master_Log_Pos:<span class="w"> </span><span class="m">236</span>
<span class="w"> </span>Relay_Log_File:<span class="w"> </span>cfhost-relay-bin.000002
<span class="w"> </span>Relay_Log_Pos:<span class="w"> </span><span class="m">235</span>
<span class="w"> </span>Relay_Master_Log_File:<span class="w"> </span>mysql-bin.000003
<span class="w"> </span>Slave_IO_Running:<span class="w"> </span>Yes
<span class="w"> </span>Slave_SQL_Running:<span class="w"> </span>Yes
<span class="w"> </span>Replicate_Do_DB:
<span class="w"> </span>Replicate_Ignore_DB:
<span class="w"> </span>Replicate_Do_Table:
<span class="w"> </span>Replicate_Ignore_Table:
<span class="w"> </span>Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
<span class="w"> </span>Last_Errno:<span class="w"> </span><span class="m">0</span>
<span class="w"> </span>Last_Error:
<span class="w"> </span>Skip_Counter:<span class="w"> </span><span class="m">0</span>
<span class="w"> </span>Exec_Master_Log_Pos:<span class="w"> </span><span class="m">236</span>
<span class="w"> </span>Relay_Log_Space:<span class="w"> </span><span class="m">235</span>
<span class="w"> </span>Until_Condition:<span class="w"> </span>None
<span class="w"> </span>Until_Log_File:
<span class="w"> </span>Until_Log_Pos:<span class="w"> </span><span class="m">0</span>
<span class="w"> </span>Master_SSL_Allowed:<span class="w"> </span>No
<span class="w"> </span>Master_SSL_CA_File:
<span class="w"> </span>Master_SSL_CA_Path:
<span class="w"> </span>Master_SSL_Cert:
<span class="w"> </span>Master_SSL_Cipher:
<span class="w"> </span>Master_SSL_Key:
<span class="w"> </span>Seconds_Behind_Master:<span class="w"> </span><span class="m">0</span>
<span class="m">1</span><span class="w"> </span>row<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="nb">set</span><span class="w"> </span><span class="o">(</span><span class="m">0</span>.00<span class="w"> </span>sec<span class="o">)</span>
ERROR:
No<span class="w"> </span>query<span class="w"> </span>specified
</code></pre></div>
<p>如果show slave status \G;<span style="color: #ff0000;">Slave_SQL_Running: No</span><span style="color: #000000;">,则执在从库上执行下面命令(两个参数值通过在主库执行show master status; 命令查看获得):
</span></p>
<div class="highlight"><pre><span></span><code>mysql<span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>stop<span class="w"> </span>slave<span class="p">;</span>
Query<span class="w"> </span>OK,<span class="w"> </span><span class="m">0</span><span class="w"> </span>rows<span class="w"> </span>affected<span class="w"> </span><span class="o">(</span><span class="m">0</span>.00<span class="w"> </span>sec<span class="o">)</span>
mysql<span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>change<span class="w"> </span>master<span class="w"> </span>to<span class="w"> </span><span class="nv">master_log_file</span><span class="o">=</span><span class="s1">'mysql-bin.000003'</span>,master_log_pos<span class="o">=</span><span class="m">236</span><span class="p">;</span>
Query<span class="w"> </span>OK,<span class="w"> </span><span class="m">0</span><span class="w"> </span>rows<span class="w"> </span>affected<span class="w"> </span><span class="o">(</span><span class="m">0</span>.01<span class="w"> </span>sec<span class="o">)</span>
</code></pre></div>
<p> </p>
<p>在主库上创建一个数据库看看是否同步.</p>
<h3 id="heartbeat">二、安装部署heartbeat实现双机热备份</h3>
<h3 id="_1">安装依赖</h3>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>pkgconfig<span class="w"> </span>glib2-devel<span class="w"> </span>python-devel<span class="w"> </span>pam-devel<span class="w"> </span>gnutls-devel<span class="w"> </span>swig
</code></pre></div>
<p>安装libnet</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://download.fedora.redhat.com/pub/epel/5/i386/libnet-1.1.5-1.el5.i386.rpm
rpm<span class="w"> </span>-ivh<span class="w"> </span>libnet-1.1.5-1.el5.i386.rpm
wget<span class="w"> </span>http://download.fedora.redhat.com/pub/epel/5/i386/libnet-devel-1.1.5-1.el5.i386.rpm
rpm<span class="w"> </span>-ivh<span class="w"> </span>libnet-devel-1.1.5-1.el5.i386.rpm
</code></pre></div>
<h3 id="_2">安装:</h3>
<div class="highlight"><pre><span></span><code>useradd<span class="w"> </span>-M<span class="w"> </span>-s<span class="w"> </span>/sbin/nologin<span class="w"> </span>hacluster
useradd<span class="w"> </span>-M<span class="w"> </span>-s<span class="w"> </span>/sbin/nologin<span class="w"> </span>haclient
wget<span class="w"> </span>http://www.ultramonkey.org/download/heartbeat/2.0.8/heartbeat-2.0.8.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>heartbeat-2.0.8.tar.gz
<span class="nb">cd</span><span class="w"> </span>heartbeat-2.0.8
./configure<span class="w"> </span>--sysconfdir<span class="o">=</span>/etc
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>创建配置文件:
安装后要配置三个文件(如没有可手动建立):ha.cf、haresources、authkeys。这三个配置文件需要在/etc/ha.d目录下面,但是默认是没有这三个文件的,可以到官网上下这三个文件,也可以在源码包里找这三个文件,在源码目录下的DOC子目录里。</p>
<div class="highlight"><pre><span></span><code>cat<span class="w"> </span>/usr/local/share/doc/heartbeat-2.0.8/ha.cf<span class="w"> </span><span class="p">|</span><span class="w"> </span>egrep<span class="w"> </span>-v<span class="w"> </span><span class="s1">'^#\W'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span><span class="s1">'^#$'</span><span class="w"> </span><span class="p">&</span>gt<span class="p">;&</span>gt<span class="p">;</span><span class="w"> </span>/etc/ha.d/ha.cf
cat<span class="w"> </span>/usr/local/share/doc/heartbeat-2.0.8/haresources<span class="w"> </span><span class="p">|</span><span class="w"> </span>egrep<span class="w"> </span>-v<span class="w"> </span><span class="s1">'^#\W'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span><span class="s1">'^#$'</span><span class="w"> </span><span class="p">&</span>gt<span class="p">;&</span>gt<span class="p">;</span><span class="w"> </span>/etc/ha.d/haresources
cat<span class="w"> </span>/usr/local/share/doc/heartbeat-2.0.8/authkeys<span class="w"> </span><span class="p">|</span><span class="w"> </span>egrep<span class="w"> </span>-v<span class="w"> </span><span class="s1">'^#\W'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="s1">'^#$'</span><span class="w"> </span>-v<span class="w"> </span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>/etc/ha.d/authkeys
</code></pre></div>
<p>编辑配置文件:</p>
<p>编辑ha.cf,该文件中包括为Heartbeat使用何种介质通路和如何配置他们的信息.</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span>vi<span class="w"> </span>/etc/ha.d/ha.cf<span class="w"> </span>
debugfile<span class="w"> </span>/var/log/ha-debug<span class="w"> </span><span class="c1"># 用于记录heartbeat的调试信息</span>
logfile<span class="w"> </span>/var/log/ha-log<span class="w"> </span><span class="c1"># 用于记录heartbeat的日志信息</span>
logfacility<span class="w"> </span>local0
keepalive<span class="w"> </span><span class="m">2</span><span class="w"> </span><span class="c1"># 设置心跳间隔</span>
watchdog<span class="w"> </span>/dev/watchdog
deadtime<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="c1"># 在30秒后宣布节点死亡</span>
warntime<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="c1"># 在日志中发出“late heartbeat“警告之前等待的时间,单位为秒</span>
initdead<span class="w"> </span><span class="m">120</span><span class="w"> </span><span class="c1"># 网络启动时间</span>
udpport<span class="w"> </span><span class="m">694</span><span class="w"> </span><span class="c1"># 广播/单播通讯使用的udp端口</span>
<span class="c1">#baud 19200</span>
<span class="c1">#serial /dev/ttyS0 # 使用串口heartbeat</span>
bcast<span class="w"> </span>eth0<span class="w"> </span><span class="c1"># 使用网卡heartbeat,并在eth0接口上使用广播heartbeat</span>
auto_failback<span class="w"> </span>on<span class="w"> </span><span class="c1"># 当主节点从故障中恢复时,将自动切换到主节点</span>
watchdog<span class="w"> </span>/dev/watchdog<span class="w"> </span><span class="c1"># 该指令是用于设置看门狗定时器,如果节点一分钟内都没有心跳,那么节点将重新启动</span>
node<span class="w"> </span>master.org<span class="w"> </span><span class="c1"># 集群中机器的主机名,与“uname –n”的输出相同。</span>
node<span class="w"> </span>slave.org
ping<span class="w"> </span><span class="m">192</span>.168.3.254<span class="w"> </span><span class="c1"># ping网关来检测链路正常</span>
respawn<span class="w"> </span>hacluster<span class="w"> </span>/usr/local/lib/heartbeat/ipfail<span class="w"> </span><span class="c1"># respawn调用/usr/lib/heartbeat/ipfail来主动进行切换</span>
apiauth<span class="w"> </span>ipfail<span class="w"> </span><span class="nv">gid</span><span class="o">=</span>haclient<span class="w"> </span><span class="nv">uid</span><span class="o">=</span>hacluster<span class="w"> </span><span class="c1"># 设置启动ipfail的用户和组</span>
</code></pre></div>
<p>配置haresources ,该文件列出所有节点所提供的服务以及服务的默认所有者.所有节点上的该文件必须相同</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/ha.d/haresources
master.org<span class="w"> </span>IPaddr::192.168.3.103<span class="w"> </span>mysql<span class="w"> </span><span class="c1"># vip</span>
</code></pre></div>
<p><code>注意:!!</code>haresources最后一个字段是某个服务的心跳,如果mysql,如果主从库使用的是同一台盘阵或者一个分布式文件系统,这里一定要填写真实的启动脚本(/etc/init.d下),如果是主从同步的话请务必不填写真正的启动脚本,因为主库心跳存活的话heartbeat会自动停止从库的mysql,这样就无法同步,主库发生故障时转移故障就没有意义.</p>
<p>配置authkeys, authkeys决定了您的认证密钥。共有三种认证方式:crc,md5,和sha1果您的Heartbeat运行于 安全 网络之上,如本例中的交叉线,可以使用crc,从资源的角度来看,这是代价最低的方法。如果网络并不 安全 ,但您也希望降低CPU使用,则使用md5。最后,如果您想得到最好的认证,而不考虑CPU使用情况,则使用sha1,它在三者之中最难破解。</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/ha.d/authkeys
auth<span class="w"> </span><span class="m">1</span>
<span class="m">1</span><span class="w"> </span>crc
chmod<span class="w"> </span><span class="m">600</span><span class="w"> </span>/etc/ha.d/authkeys
</code></pre></div>
<p>不论您在关键字auth后面指定的是什么索引值,在后面必须要作为键值再次出现。如果您指定“auth 4”,则在后面一定要有一行的内容为“4 ”。
配置从库:</p>
<div class="highlight"><pre><span></span><code>scp<span class="w"> </span>root@192.168.3.101:/etc/ha.d/ha.cf<span class="w"> </span>/etc/ha.d/
scp<span class="w"> </span>root@192.168.3.101:/etc/ha.d/authkeys<span class="w"> </span>/etc/ha.d/
scp<span class="w"> </span>root@192.168.3.101:/etc/ha.d/haresources<span class="w"> </span>/etc/ha.d/
vi<span class="w"> </span>/etc/ha.d/ha.cf
debugfile<span class="w"> </span>/var/log/ha-debug
logfile<span class="w"> </span>/var/log/ha-log
logfacility<span class="w"> </span>local0
keepalive<span class="w"> </span><span class="m">2</span>
deadtime<span class="w"> </span><span class="m">30</span>
warntime<span class="w"> </span><span class="m">10</span>
initdead<span class="w"> </span><span class="m">120</span>
udpport<span class="w"> </span><span class="m">694</span>
bcast<span class="w"> </span>eth0<span class="w"> </span>
auto_failback<span class="w"> </span>on
node<span class="w"> </span>master.org
node<span class="w"> </span>slave.org
ping<span class="w"> </span><span class="m">192</span>.168.3.254
respawn<span class="w"> </span>hacluser<span class="w"> </span>/usr/local/lib/heartbeat/ipfail<span class="w"> </span><span class="c1"># respawn调用/usr/lib/heartbeat/ipfail来主动进行切换</span>
apiauth<span class="w"> </span>ipfail<span class="w"> </span><span class="nv">gid</span><span class="o">=</span>haclient<span class="w"> </span><span class="nv">uid</span><span class="o">=</span>hacluster
</code></pre></div>
<p>启动主库heartbeat:</p>
<div class="highlight"><pre><span></span><code>server<span class="w"> </span>heartbeat<span class="w"> </span>start
</code></pre></div>
<p>查看日志:</p>
<div class="highlight"><pre><span></span><code>cat<span class="w"> </span>/var/log/ha-log
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:29<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span><span class="m">192</span>.168.3.254:192.168.3.254<span class="w"> </span>up.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:29<span class="w"> </span>info:<span class="w"> </span>Status<span class="w"> </span>update<span class="w"> </span><span class="k">for</span><span class="w"> </span>node<span class="w"> </span><span class="m">192</span>.168.3.254:<span class="w"> </span>status<span class="w"> </span>ping
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:29<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span>master.org:eth0<span class="w"> </span>up.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>WARN:<span class="w"> </span>node<span class="w"> </span>slave.org:<span class="w"> </span>is<span class="w"> </span>dead
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>info:<span class="w"> </span>Comm_now_up<span class="o">()</span>:<span class="w"> </span>updating<span class="w"> </span>status<span class="w"> </span>to<span class="w"> </span>active
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>status<span class="w"> </span>now<span class="w"> </span><span class="nb">set</span><span class="w"> </span>to:<span class="w"> </span><span class="s1">'active'</span>
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>info:<span class="w"> </span>Starting<span class="w"> </span>child<span class="w"> </span>client<span class="w"> </span><span class="s2">"/usr/local/lib/heartbeat/ipfail"</span><span class="w"> </span><span class="o">(</span><span class="m">503</span>,503<span class="o">)</span>
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>WARN:<span class="w"> </span>No<span class="w"> </span>STONITH<span class="w"> </span>device<span class="w"> </span>configured.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>WARN:<span class="w"> </span>Shared<span class="w"> </span>disks<span class="w"> </span>are<span class="w"> </span>not<span class="w"> </span>protected.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>info:<span class="w"> </span>Resources<span class="w"> </span>being<span class="w"> </span>acquired<span class="w"> </span>from<span class="w"> </span>slave.org.
heartbeat<span class="o">[</span><span class="m">32247</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:41<span class="w"> </span>info:<span class="w"> </span>Starting<span class="w"> </span><span class="s2">"/usr/local/lib/heartbeat/ipfail"</span><span class="w"> </span>as<span class="w"> </span>uid<span class="w"> </span><span class="m">503</span><span class="w"> </span>gid<span class="w"> </span><span class="m">503</span><span class="w"> </span><span class="o">(</span>pid<span class="w"> </span><span class="m">32247</span><span class="o">)</span>
harc<span class="o">[</span><span class="m">32248</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/rc.d/status<span class="w"> </span>status
mach_down<span class="o">[</span><span class="m">32275</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>/usr/local/lib/heartbeat/mach_down:<span class="w"> </span>nice_failback:<span class="w"> </span>foreign<span class="w"> </span>resources<span class="w"> </span>acquired
mach_down<span class="o">[</span><span class="m">32275</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>mach_down<span class="w"> </span>takeover<span class="w"> </span><span class="nb">complete</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>node<span class="w"> </span>slave.org.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>mach_down<span class="w"> </span>takeover<span class="w"> </span>complete.
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Initial<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span><span class="nb">complete</span><span class="w"> </span><span class="o">(</span>mach_down<span class="o">)</span>
IPaddr<span class="o">[</span><span class="m">32300</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>INFO:<span class="w"> </span>Resource<span class="w"> </span>is<span class="w"> </span>stopped
heartbeat<span class="o">[</span><span class="m">32249</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>Resource<span class="w"> </span>acquisition<span class="w"> </span>completed.
harc<span class="o">[</span><span class="m">32338</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/rc.d/ip-request-resp<span class="w"> </span>ip-request-resp
ip-request-resp<span class="o">[</span><span class="m">32338</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>received<span class="w"> </span>ip-request-resp<span class="w"> </span>IPaddr::192.168.3.103<span class="w"> </span>OK<span class="w"> </span>yes
ResourceManager<span class="o">[</span><span class="m">32353</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Acquiring<span class="w"> </span>resource<span class="w"> </span>group:<span class="w"> </span>master.org<span class="w"> </span>IPaddr::192.168.3.103<span class="w"> </span>mysqld
IPaddr<span class="o">[</span><span class="m">32377</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>INFO:<span class="w"> </span>Resource<span class="w"> </span>is<span class="w"> </span>stopped
ResourceManager<span class="o">[</span><span class="m">32353</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/resource.d/IPaddr<span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>start
IPaddr<span class="o">[</span><span class="m">32429</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>INFO:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>nic<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span>eth0
IPaddr<span class="o">[</span><span class="m">32429</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>DEBUG:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>netmask<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span><span class="m">255</span>.255.255.0
IPaddr<span class="o">[</span><span class="m">32429</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>DEBUG:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>broadcast<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span><span class="m">192</span>.168.3.255
IPaddr<span class="o">[</span><span class="m">32429</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:42<span class="w"> </span>INFO:<span class="w"> </span><span class="nb">eval</span><span class="w"> </span>/sbin/ifconfig<span class="w"> </span>eth0:0<span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>netmask<span class="w"> </span><span class="m">255</span>.255.255.0<span class="w"> </span>broadcast<span class="w"> </span><span class="m">192</span>.168.3.255
IPaddr<span class="o">[</span><span class="m">32429</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:43<span class="w"> </span>DEBUG:<span class="w"> </span>Sending<span class="w"> </span>Gratuitous<span class="w"> </span>Arp<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>on<span class="w"> </span>eth0:0<span class="w"> </span><span class="o">[</span>eth0<span class="o">]</span>
IPaddr<span class="o">[</span><span class="m">32420</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:43<span class="w"> </span>INFO:<span class="w"> </span>Success
ResourceManager<span class="o">[</span><span class="m">32353</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:43<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span>start
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:56<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>Resource<span class="w"> </span>acquisition<span class="w"> </span>completed.<span class="w"> </span><span class="o">(</span>none<span class="o">)</span>
heartbeat<span class="o">[</span><span class="m">32239</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_13:45:56<span class="w"> </span>info:<span class="w"> </span><span class="nb">local</span><span class="w"> </span>resource<span class="w"> </span>transition<span class="w"> </span>completed.
</code></pre></div>
<p> </p>
<p>从日志中看出来slave.org没起来是死亡的,并添加192.168.3.103vip</p>
<p>启动从库heartbeat</p>
<div class="highlight"><pre><span></span><code>server<span class="w"> </span>heartbeat<span class="w"> </span>start
</code></pre></div>
<p>启动之后查看日志信息</p>
<div class="highlight"><pre><span></span><code>Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:22<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>status<span class="w"> </span>now<span class="w"> </span><span class="nb">set</span><span class="w"> </span>to:<span class="w"> </span><span class="s1">'up'</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span>master.org:eth0<span class="w"> </span>up.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Status<span class="w"> </span>update<span class="w"> </span><span class="k">for</span><span class="w"> </span>node<span class="w"> </span>master.org:<span class="w"> </span>status<span class="w"> </span>active
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span><span class="m">192</span>.168.3.254:192.168.3.254<span class="w"> </span>up.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Status<span class="w"> </span>update<span class="w"> </span><span class="k">for</span><span class="w"> </span>node<span class="w"> </span><span class="m">192</span>.168.3.254:<span class="w"> </span>status<span class="w"> </span>ping
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span>slave.org:eth0<span class="w"> </span>up.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:23<span class="w"> </span>slave<span class="w"> </span>harc<span class="o">[</span><span class="m">29171</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/rc.d/status<span class="w"> </span>status
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Comm_now_up<span class="o">()</span>:<span class="w"> </span>updating<span class="w"> </span>status<span class="w"> </span>to<span class="w"> </span>active
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>status<span class="w"> </span>now<span class="w"> </span><span class="nb">set</span><span class="w"> </span>to:<span class="w"> </span><span class="s1">'active'</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Starting<span class="w"> </span>child<span class="w"> </span>client<span class="w"> </span><span class="s2">"/usr/local/lib/heartbeat/ipfail"</span><span class="w"> </span><span class="o">(</span><span class="m">501</span>,501<span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>WARN:<span class="w"> </span>G_CH_dispatch_int:<span class="w"> </span>Dispatch<span class="w"> </span><span class="k">function</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nb">read</span><span class="w"> </span>child<span class="w"> </span>took<span class="w"> </span>too<span class="w"> </span>long<span class="w"> </span>to<span class="w"> </span>execute:<span class="w"> </span><span class="m">140</span><span class="w"> </span>ms<span class="w"> </span><span class="o">(</span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span><span class="m">50</span><span class="w"> </span>ms<span class="o">)</span><span class="w"> </span><span class="o">(</span>GSource:<span class="w"> </span>0x9b98448<span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29182</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Starting<span class="w"> </span><span class="s2">"/usr/local/lib/heartbeat/ipfail"</span><span class="w"> </span>as<span class="w"> </span>uid<span class="w"> </span><span class="m">501</span><span class="w"> </span>gid<span class="w"> </span><span class="m">501</span><span class="w"> </span><span class="o">(</span>pid<span class="w"> </span><span class="m">29182</span><span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>remote<span class="w"> </span>resource<span class="w"> </span>transition<span class="w"> </span>completed.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>remote<span class="w"> </span>resource<span class="w"> </span>transition<span class="w"> </span>completed.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:24<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>Resource<span class="w"> </span>acquisition<span class="w"> </span>completed.<span class="w"> </span><span class="o">(</span>none<span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:25<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>master.org<span class="w"> </span>wants<span class="w"> </span>to<span class="w"> </span>go<span class="w"> </span>standby<span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:26<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>standby:<span class="w"> </span>acquire<span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span><span class="w"> </span>resources<span class="w"> </span>from<span class="w"> </span>master.org
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:26<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29183</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>acquire<span class="w"> </span><span class="nb">local</span><span class="w"> </span>HA<span class="w"> </span>resources<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:26<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29183</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span><span class="nb">local</span><span class="w"> </span>HA<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span>completed<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:26<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Standby<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span><span class="k">done</span><span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:26<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Initial<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span><span class="nb">complete</span><span class="w"> </span><span class="o">(</span>auto_failback<span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:27<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>remote<span class="w"> </span>resource<span class="w"> </span>transition<span class="w"> </span>completed.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:36<span class="w"> </span>slave<span class="w"> </span>ipfail:<span class="w"> </span><span class="o">[</span><span class="m">29182</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Ping<span class="w"> </span>node<span class="w"> </span>count<span class="w"> </span>is<span class="w"> </span>balanced.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:37<span class="w"> </span>slave<span class="w"> </span>ipfail:<span class="w"> </span><span class="o">[</span><span class="m">29182</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Giving<span class="w"> </span>up<span class="w"> </span>foreign<span class="w"> </span>resources<span class="w"> </span><span class="o">(</span>auto_failback<span class="o">)</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:37<span class="w"> </span>slave<span class="w"> </span>ipfail:<span class="w"> </span><span class="o">[</span><span class="m">29182</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Delayed<span class="w"> </span>giveup<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">4</span><span class="w"> </span>seconds.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:42<span class="w"> </span>slave<span class="w"> </span>ipfail:<span class="w"> </span><span class="o">[</span><span class="m">29182</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>giveup<span class="o">()</span><span class="w"> </span>called<span class="w"> </span><span class="o">(</span>timeout<span class="w"> </span>worked<span class="o">)</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:42<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>slave.org<span class="w"> </span>wants<span class="w"> </span>to<span class="w"> </span>go<span class="w"> </span>standby<span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:43<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>standby:<span class="w"> </span>master.org<span class="w"> </span>can<span class="w"> </span>take<span class="w"> </span>our<span class="w"> </span>foreign<span class="w"> </span>resources
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:43<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29194</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>give<span class="w"> </span>up<span class="w"> </span>foreign<span class="w"> </span>HA<span class="w"> </span>resources<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:43<span class="w"> </span>slave<span class="w"> </span>ResourceManager<span class="o">[</span><span class="m">29204</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Releasing<span class="w"> </span>resource<span class="w"> </span>group:<span class="w"> </span>master.org<span class="w"> </span>IPaddr::192.168.3.103<span class="w"> </span>mysqld
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:43<span class="w"> </span>slave<span class="w"> </span>ResourceManager<span class="o">[</span><span class="m">29204</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span>stop
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:45<span class="w"> </span>slave<span class="w"> </span>ResourceManager<span class="o">[</span><span class="m">29204</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/resource.d/IPaddr<span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>stop
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:45<span class="w"> </span>slave<span class="w"> </span>IPaddr<span class="o">[</span><span class="m">29279</span><span class="o">]</span>:<span class="w"> </span>INFO:<span class="w"> </span>Success
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:45<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29194</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>foreign<span class="w"> </span>HA<span class="w"> </span>resource<span class="w"> </span>release<span class="w"> </span>completed<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:45<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Local<span class="w"> </span>standby<span class="w"> </span>process<span class="w"> </span>completed<span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span>.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:46<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>WARN:<span class="w"> </span><span class="m">1</span><span class="w"> </span>lost<span class="w"> </span>packet<span class="o">(</span>s<span class="o">)</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="o">[</span>master.org<span class="o">]</span><span class="w"> </span><span class="o">[</span><span class="m">162</span>:164<span class="o">]</span>
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:46<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>remote<span class="w"> </span>resource<span class="w"> </span>transition<span class="w"> </span>completed.
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:46<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>No<span class="w"> </span>pkts<span class="w"> </span>missing<span class="w"> </span>from<span class="w"> </span>master.org!
Feb<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">13</span>:50:46<span class="w"> </span>slave<span class="w"> </span>heartbeat:<span class="w"> </span><span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span>info:<span class="w"> </span>Other<span class="w"> </span>node<span class="w"> </span>completed<span class="w"> </span>standby<span class="w"> </span>takeover<span class="w"> </span>of<span class="w"> </span>foreign<span class="w"> </span>resources.
</code></pre></div>
<p>现在尝试停止主库的MySQL服务</p>
<div class="highlight"><pre><span></span><code>pkill<span class="w"> </span>mysqld
</code></pre></div>
<p>查看日志并无变化,所以得出结论heartbeat只检测心跳也就是只检测设备是否宕机,不会检测MySQL服务,所以我们同样要有一个脚本来检测MySQL服务,如果mysql服务宕掉,则尝试启动服务,若启动服务失败则kill掉heartbeat进程实现故障转移(和上一遍nginx+keepalived原理一致),脚本内容如下:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1"># filename:mysqlsc.sh</span>
ps<span class="w"> </span>aux<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>mysqld<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span>grep<span class="w"> </span><span class="m">2</span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">1</span><span class="p">&</span>gt<span class="p">;&</span>amp<span class="p">;</span><span class="m">2</span><span class="w"> </span><span class="c1"># 过滤mysql进程</span>
<span class="k">if</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]]</span><span class="w"> </span><span class="c1"># 如果过滤有mysql进程会返回0则认为mysql存活</span>
<span class="k">then</span>
<span class="w"> </span>sleep<span class="w"> </span><span class="m">5</span><span class="w"> </span><span class="c1"># 使脚本进入休眠</span>
<span class="k">else</span>
<span class="c1"># 如果nginx没有存活尝试启动mysql,如果失败则杀死heartbeat的进程</span>
<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span>start
<span class="w"> </span>ps<span class="w"> </span>aux<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>mysqld<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span>grep<span class="w"> </span><span class="m">2</span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">1</span><span class="p">&</span>gt<span class="p">;&</span>amp<span class="p">;</span><span class="m">2</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>pkill<span class="w"> </span>heartbeat
<span class="w"> </span><span class="k">fi</span>
<span class="k">fi</span>
</code></pre></div>
<p>给这个脚本执行权限然后后台运行:</p>
<div class="highlight"><pre><span></span><code>chmod<span class="w"> </span>+x<span class="w"> </span>mysqlsc.sh
nohup<span class="w"> </span>sh<span class="w"> </span>mysqlsc.sh<span class="w"> </span><span class="p">&</span>amp<span class="p">;</span><span class="w"> </span><span class="c1"># 后台运行</span>
</code></pre></div>
<p>下面来尝试停止主库的heartbeat:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>heartbeat<span class="w"> </span>stop
</code></pre></div>
<p>查看从库日志:</p>
<div class="highlight"><pre><span></span><code>heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Received<span class="w"> </span>shutdown<span class="w"> </span>notice<span class="w"> </span>from<span class="w"> </span><span class="s1">'master.org'</span>.
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Resources<span class="w"> </span>being<span class="w"> </span>acquired<span class="w"> </span>from<span class="w"> </span>master.org.
heartbeat<span class="o">[</span><span class="m">29308</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>acquire<span class="w"> </span><span class="nb">local</span><span class="w"> </span>HA<span class="w"> </span>resources<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
heartbeat<span class="o">[</span><span class="m">29308</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span><span class="nb">local</span><span class="w"> </span>HA<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span>completed<span class="w"> </span><span class="o">(</span>standby<span class="o">)</span>.
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Standby<span class="w"> </span>resource<span class="w"> </span>acquisition<span class="w"> </span><span class="k">done</span><span class="w"> </span><span class="o">[</span>foreign<span class="o">]</span>.
heartbeat<span class="o">[</span><span class="m">29309</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>No<span class="w"> </span><span class="nb">local</span><span class="w"> </span>resources<span class="w"> </span><span class="o">[</span>/usr/local/lib/heartbeat/ResourceManager<span class="w"> </span>listkeys<span class="w"> </span>slave.org<span class="o">]</span><span class="w"> </span>to<span class="w"> </span>acquire.
harc<span class="o">[</span><span class="m">29328</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/rc.d/status<span class="w"> </span>status
mach_down<span class="o">[</span><span class="m">29338</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Taking<span class="w"> </span>over<span class="w"> </span>resource<span class="w"> </span>group<span class="w"> </span>IPaddr::192.168.3.103
ResourceManager<span class="o">[</span><span class="m">29358</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>info:<span class="w"> </span>Acquiring<span class="w"> </span>resource<span class="w"> </span>group:<span class="w"> </span>master.org<span class="w"> </span>IPaddr::192.168.3.103<span class="w"> </span>mysqld
IPaddr<span class="o">[</span><span class="m">29382</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:05<span class="w"> </span>INFO:<span class="w"> </span>Resource<span class="w"> </span>is<span class="w"> </span>stopped
ResourceManager<span class="o">[</span><span class="m">29358</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/ha.d/resource.d/IPaddr<span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>start
IPaddr<span class="o">[</span><span class="m">29434</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>INFO:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>nic<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span>eth0
IPaddr<span class="o">[</span><span class="m">29434</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>DEBUG:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>netmask<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span><span class="m">255</span>.255.255.0
IPaddr<span class="o">[</span><span class="m">29434</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>DEBUG:<span class="w"> </span>Using<span class="w"> </span>calculated<span class="w"> </span>broadcast<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103:<span class="w"> </span><span class="m">192</span>.168.3.255
IPaddr<span class="o">[</span><span class="m">29434</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>INFO:<span class="w"> </span><span class="nb">eval</span><span class="w"> </span>/sbin/ifconfig<span class="w"> </span>eth0:0<span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>netmask<span class="w"> </span><span class="m">255</span>.255.255.0<span class="w"> </span>broadcast<span class="w"> </span><span class="m">192</span>.168.3.255
IPaddr<span class="o">[</span><span class="m">29434</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>DEBUG:<span class="w"> </span>Sending<span class="w"> </span>Gratuitous<span class="w"> </span>Arp<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.103<span class="w"> </span>on<span class="w"> </span>eth0:0<span class="w"> </span><span class="o">[</span>eth0<span class="o">]</span>
IPaddr<span class="o">[</span><span class="m">29425</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>INFO:<span class="w"> </span>Success
ResourceManager<span class="o">[</span><span class="m">29358</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:06<span class="w"> </span>info:<span class="w"> </span>Running<span class="w"> </span>/etc/init.d/mysqld<span class="w"> </span>start
mach_down<span class="o">[</span><span class="m">29338</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:07<span class="w"> </span>info:<span class="w"> </span>/usr/local/lib/heartbeat/mach_down:<span class="w"> </span>nice_failback:<span class="w"> </span>foreign<span class="w"> </span>resources<span class="w"> </span>acquired
mach_down<span class="o">[</span><span class="m">29338</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:07<span class="w"> </span>info:<span class="w"> </span>mach_down<span class="w"> </span>takeover<span class="w"> </span><span class="nb">complete</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>node<span class="w"> </span>master.org.
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:07<span class="w"> </span>info:<span class="w"> </span>mach_down<span class="w"> </span>takeover<span class="w"> </span>complete.
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:17<span class="w"> </span>WARN:<span class="w"> </span>node<span class="w"> </span>master.org:<span class="w"> </span>is<span class="w"> </span>dead
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:17<span class="w"> </span>info:<span class="w"> </span>Dead<span class="w"> </span>node<span class="w"> </span>master.org<span class="w"> </span>gave<span class="w"> </span>up<span class="w"> </span>resources.
heartbeat<span class="o">[</span><span class="m">29159</span><span class="o">]</span>:<span class="w"> </span><span class="m">2012</span>/02/19_14:03:17<span class="w"> </span>info:<span class="w"> </span>Link<span class="w"> </span>master.org:eth0<span class="w"> </span>dead.
</code></pre></div>nginx+keepalived配置高可用HTTP群集2012-02-09T17:35:00+08:002012-02-09T17:35:00+08:00coldtag:www.linuxzen.com,2012-02-09:/nginxkeepalivedpei-zhi-gao-ke-yong-httpqun-ji.html<p>Nginx不仅是一款优秀的WEB服务器,同时可以根据nginx的反代理可以配置成强大的负载均衡器.这里就介绍如何把nginx配置成负载均衡器,并结合keepalived配置高可用的集群.
一般集群主要架构为:</p>
<p>前端为负载均衡器两个:主/备,两种工作方式,一种 …</p><p>Nginx不仅是一款优秀的WEB服务器,同时可以根据nginx的反代理可以配置成强大的负载均衡器.这里就介绍如何把nginx配置成负载均衡器,并结合keepalived配置高可用的集群.
一般集群主要架构为:</p>
<p>前端为负载均衡器两个:主/备,两种工作方式,一种是备机待机状态,主机故障时备机接管主机工作实现故障庄毅,在主机故障恢复完成时备机继续仅需待机状态,第二种是主备同时工作,一台宕机另外一台自动接管另一台的工作实现故障转移.
第一种方式可以通过将域名解析到一个虚拟ip(vip)上,主负载均衡器绑定虚拟ip,当主负载均衡器出现故障时,通过keepalived自动将vip绑定到备用负载均衡器上同时arping网关刷新MAC地址.,避免单点故障.
第二种方式主备同时绑定一个vip,把域名通过DNS轮询的方式解析到这两个服务器上,主机出现故障,备机就将主机绑定vip绑定到备机上,同时arping网关刷新MAC地址.实现故障转移.</p>
<p>中间为WEB服务器作为real server,处理请求.
后端为数据库和分布式文件系统.数据库一般为主从两台.分布式文件系统有效解决WEB服务器之间的数据同步.有的还会将图片服务器单独分离出来放在后端.</p>
<p>本文使用环境:</p>
<ul>
<li>CentOS 5.5 32位</li>
<li>nginx:nginx-1.0.11</li>
<li>keepalived:keepalived-1.1.19.tar.gz</li>
<li>主调度器:192.168.3.1</li>
<li>备调度器:192.168.3.2</li>
<li>real server:192.168.3.4/5/6</li>
</ul>
<p>本文采用第一种方式来进行vip为:192.168.3.253</p>
<h2 id="nginx">一、在主备服务器上部署nginx</h2>
<h3 id="1">1.下载</h3>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>http://nginx.org/download/nginx-1.0.11.tar.gz
</code></pre></div>
<h3 id="2">2.安装</h3>
<div class="highlight"><pre><span></span><code><span class="w"> </span>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>zlib-devel<span class="w"> </span>pcre-devel<span class="w"> </span>openssl-devel<span class="w"> </span><span class="c1"># 安装依赖</span>
tar<span class="w"> </span>-zxvf<span class="w"> </span>nginx-1.0.11.tar.gz
<span class="nb">cd</span><span class="w"> </span>nginx-1.0.11
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/nginx<span class="w"> </span>--with-http_ssl_module<span class="w"> </span>--with-http_flv_module<span class="w"> </span>--with-http_gzip_static_module
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>3.配置</p>
<p>配置主调度器的nginx,编辑nginx.conf</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/usr/local/nginx/conf/nginx.conf
http<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>include<span class="w"> </span>mime.types<span class="p">;</span>
<span class="w"> </span>default_type<span class="w"> </span>application/octet-stream<span class="p">;</span>
<span class="w"> </span><span class="c1">#log_format main '$remote_addr - $remote_user [$time_local] "$request" '</span>
<span class="w"> </span><span class="c1"># '$status $body_bytes_sent "$http_referer" '</span>
<span class="w"> </span><span class="c1"># '"$http_user_agent" "$http_x_forwarded_for"';</span>
<span class="w"> </span><span class="c1">#access_log logs/access.log main;</span>
<span class="w"> </span>sendfile<span class="w"> </span>on<span class="p">;</span>
<span class="w"> </span><span class="c1">#tcp_nopush on;</span>
<span class="w"> </span><span class="c1">#keepalive_timeout 0;</span>
<span class="w"> </span>keepalive_timeout<span class="w"> </span><span class="m">65</span><span class="p">;</span>
<span class="w"> </span><span class="c1">#gzip on;</span>
<span class="w"> </span><span class="c1"># 添加一组真实的服务器地址池</span>
<span class="w"> </span><span class="c1"># 供proxy_pass和fastcgi_pass指令中使用的代理服务器</span>
<span class="w"> </span>upstream<span class="w"> </span>real_server_pool<span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="c1"># 后台如果有动态应用的时候,ip_hash指令可以通过hash算法</span>
<span class="w"> </span><span class="c1"># 将客户端请求定位到同一台后端服务器上,解决session共享,</span>
<span class="w"> </span><span class="c1"># 但建议用动态应用做session共享</span>
<span class="w"> </span><span class="c1"># ip_hash;</span>
<span class="w"> </span><span class="c1"># server用于指定一个后端服务器的名称和参数</span>
<span class="w"> </span><span class="c1"># weight代表权,重默认为1,权重越高被分配的客户端越多</span>
<span class="w"> </span><span class="c1"># max_fails 指定时间内对后端请求失败的次数</span>
<span class="w"> </span><span class="c1"># fail_timeout 达到max_fails指定的失败次数后暂停的时间</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.4:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="p">;</span>
<span class="w"> </span><span class="c1"># down参数用来标记为离线,不参与负载均衡.在ip_hash下使用</span>
<span class="w"> </span><span class="c1"># 在此做演示,后面测试会去掉</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.5:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="w"> </span>down<span class="p">;</span>
<span class="w"> </span><span class="c1"># backup仅仅在非backup服务器宕机或繁忙的时候使用</span>
<span class="w"> </span><span class="c1"># (在此做演示,后面测试会去掉)</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.6:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="w"> </span>backup<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.1:80<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span>localhost<span class="p">;</span>
<span class="w"> </span><span class="c1">#charset koi8-r;</span>
<span class="w"> </span><span class="c1">#access_log logs/host.access.log main;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="c1">#root html;</span>
<span class="w"> </span><span class="c1">#index index.html index.htm;</span>
<span class="w"> </span><span class="c1"># 使用upstream设置的一组代理服务器</span>
<span class="w"> </span><span class="c1"># 如果后端服务器出现502或504等执行错误时,</span>
<span class="w"> </span><span class="c1"># 将自动将请求转发给负载均衡池中的另一台服务器.</span>
<span class="w"> </span>proxy_next_upstream<span class="w"> </span>http_502<span class="w"> </span>http_504<span class="w"> </span>error<span class="w"> </span>timeout<span class="w"> </span>invalid_header<span class="p">;</span>
<span class="w"> </span>proxy_pass<span class="w"> </span>http://real_server_pool<span class="p">;</span>
<span class="w"> </span>proxy_set_header<span class="w"> </span>Host<span class="w"> </span><span class="nv">$host</span><span class="p">;</span>
<span class="w"> </span>proxy_set_header<span class="w"> </span>X-Forwarded-For<span class="w"> </span><span class="nv">$remote_addr</span><span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
</code></pre></div>
<p>(<code>注意:</code>配置文件中注释ip_hash,以为ip_hash这个功能将保证这个客户端请求总是被转发到一台服务器上,所以如果启用了ip_hash指令,将不能再使用weight(权重参数),配置文件中加入为解释ip_hash指令)
配置备用nginx,将监听ip改为备用调度器的ip</p>
<div class="highlight"><pre><span></span><code>http<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>include<span class="w"> </span>mime.types<span class="p">;</span>
<span class="w"> </span>default_type<span class="w"> </span>application/octet-stream<span class="p">;</span>
<span class="w"> </span><span class="c1">#log_format main '$remote_addr - $remote_user [$time_local] "$request" '</span>
<span class="w"> </span><span class="c1"># '$status $body_bytes_sent "$http_referer" '</span>
<span class="w"> </span><span class="c1"># '"$http_user_agent" "$http_x_forwarded_for"';</span>
<span class="w"> </span><span class="c1">#access_log logs/access.log main;</span>
<span class="w"> </span>sendfile<span class="w"> </span>on<span class="p">;</span>
<span class="w"> </span><span class="c1">#tcp_nopush on;</span>
<span class="w"> </span><span class="c1">#keepalive_timeout 0;</span>
<span class="w"> </span>keepalive_timeout<span class="w"> </span><span class="m">65</span><span class="p">;</span>
<span class="w"> </span><span class="c1">#gzip on;</span>
<span class="w"> </span>upstream<span class="w"> </span>real_server_pool<span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="c1">#ip_hash;</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.4:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="p">;</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.5:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="p">;</span>
<span class="w"> </span>server<span class="w"> </span><span class="m">192</span>.168.3.6:80<span class="w"> </span><span class="nv">weight</span><span class="o">=</span><span class="m">1</span><span class="w"> </span><span class="nv">max_fails</span><span class="o">=</span><span class="m">2</span><span class="w"> </span><span class="nv">fail_timeout</span><span class="o">=</span>30s<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.2:80<span class="p">;</span><span class="w"> </span><span class="c1"># 监听ip改为本地ip</span>
<span class="w"> </span>server_name<span class="w"> </span>localhost<span class="p">;</span>
<span class="w"> </span><span class="c1">#charset koi8-r;</span>
<span class="w"> </span><span class="c1">#access_log logs/host.access.log main;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="c1">#root html;</span>
<span class="w"> </span><span class="c1">#index index.html index.htm;</span>
<span class="w"> </span>proxy_next_upstream<span class="w"> </span>http_502<span class="w"> </span>http_504<span class="w"> </span>error<span class="w"> </span>timeout<span class="w"> </span>invalid_header<span class="p">;</span>
<span class="w"> </span>proxy_pass<span class="w"> </span>http://real_server_pool<span class="p">;</span>
<span class="w"> </span>proxy_set_header<span class="w"> </span>Host<span class="w"> </span><span class="nv">$host</span><span class="p">;</span>
<span class="w"> </span>proxy_set_header<span class="w"> </span>X-Forwarded-For<span class="w"> </span><span class="nv">$remote_addr</span><span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
</code></pre></div>
<p>然后启动主备nginx:</p>
<div class="highlight"><pre><span></span><code>/usr/local/nginx/sbin/nginx
</code></pre></div>
<h2 id="keepalived">二、在主备服务器上部署keepalived</h2>
<p>安装
安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>kernel-devel<span class="w"> </span><span class="c1"># 安装依赖</span>
</code></pre></div>
<p>开启路由转发:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/sysctl.conf
net.ipv4.ip_forward<span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># 此参数改为1</span>
sysctl<span class="w"> </span>-p<span class="w"> </span><span class="c1"># 使修改生效</span>
</code></pre></div>
<p>首先安装ipvs:</p>
<div class="highlight"><pre><span></span><code>ln<span class="w"> </span>-s<span class="w"> </span>/usr/src/kernels/2.6.18-194.el5-i686/<span class="w"> </span>/usr/src/linux<span class="w"> </span><span class="c1"># ipvs需要内核文件,做一个软连接</span>
<span class="c1"># 下载</span>
wget<span class="w"> </span>http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>ipvsadm-1.24.tar.gz
<span class="nb">cd</span><span class="w"> </span>ipvsadm-1.24
make
make<span class="w"> </span>install
</code></pre></div>
<p>然后安装keepalived</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 下载</span>
wget<span class="w"> </span>http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
tar<span class="w"> </span>-zxvf<span class="w"> </span>keepalived-1.1.19.tar.gz
<span class="nb">cd</span><span class="w"> </span>keepalived-1.1.19
./configure<span class="w"> </span>--prefix<span class="o">=</span>/<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 安装在默认位置(配置文件,二进制文件,启动脚本放到默认位置)</span>
--mandir<span class="o">=</span>/usr/local/share/man/<span class="w"> </span><span class="se">\</span>
--with-kernel-dir<span class="o">=</span>/usr/src/kernels/2.6.18-194.el5-i686/<span class="w"> </span><span class="c1"># 需要内核的头文件</span>
make<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<h3 id="keepalived_1">配置keepalived</h3>
<p>编辑主调度器配置文件/etc/keepalived/keepalived.conf ###</p>
<div class="highlight"><pre><span></span><code>global_defs<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>notification_email<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>cold_night@linuxzen.com<span class="w"> </span><span class="c1"># 定义通知邮箱,有多个可以换行添加</span>
<span class="o">}</span>
<span class="w"> </span>notification_email_from<span class="w"> </span>root@linuxzen.com#<span class="w"> </span>定义发送邮件的邮箱
<span class="w"> </span>smtp_server<span class="w"> </span>www.linuxzen.com<span class="w"> </span><span class="c1"># 定义发件服务器</span>
<span class="w"> </span>smtp_connect_timeout<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="c1"># 定义连接smtp服务器超时时间</span>
<span class="w"> </span>router_id<span class="w"> </span>LVS_DEVEL
<span class="o">}</span>
vrrp_instance<span class="w"> </span>VI_1<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>state<span class="w"> </span>MASTER<span class="w"> </span><span class="c1"># 标示主备,备机上改为BACKUP</span>
<span class="w"> </span>interface<span class="w"> </span>eth0<span class="w"> </span><span class="c1"># HA监测的端口</span>
<span class="w"> </span>virtual_router_id<span class="w"> </span><span class="m">51</span><span class="w"> </span><span class="c1"># 主备的virtual_router_id的值必须相同</span>
<span class="w"> </span>priority<span class="w"> </span><span class="m">100</span><span class="w"> </span><span class="c1"># 优先级,通常主要比备稍大</span>
<span class="w"> </span>advert_int<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># VRRP Multicast 广播周期秒数</span>
<span class="w"> </span>authentication<span class="w"> </span><span class="o">{</span><span class="w"> </span><span class="c1"># 定义认证</span>
<span class="w"> </span>auth_type<span class="w"> </span>PASS<span class="w"> </span><span class="c1"># 认证方式</span>
<span class="w"> </span>auth_pass<span class="w"> </span><span class="m">1111</span><span class="w"> </span><span class="c1"># 认证口令字</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>virtual_ipaddress<span class="w"> </span><span class="o">{</span><span class="w"> </span><span class="c1"># 定义vip</span>
<span class="w"> </span><span class="m">192</span>.168.3.253<span class="w"> </span><span class="c1"># 多个可换行添加,一行一个</span>
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
virtual_server<span class="w"> </span><span class="m">192</span>.168.3.253<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>delay_loop<span class="w"> </span><span class="m">6</span><span class="w"> </span><span class="c1"># 每隔 6 秒查询 realserver 状态</span>
<span class="w"> </span>lb_algo<span class="w"> </span>rr
<span class="w"> </span>lb_kind<span class="w"> </span>NAT
<span class="w"> </span>nat_mask<span class="w"> </span><span class="m">255</span>.255.255.0
<span class="w"> </span>persistence_timeout<span class="w"> </span><span class="m">50</span><span class="w"> </span><span class="c1"># 同一IP 的连接50秒内被分配到同一台realserver</span>
<span class="w"> </span>protocol<span class="w"> </span>TCP<span class="w"> </span><span class="c1"># 用TCP监测realserver的状态</span>
<span class="w"> </span>real_server<span class="w"> </span><span class="m">192</span>.168.3.1<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>weight<span class="w"> </span><span class="m">3</span><span class="w"> </span><span class="c1"># 权重</span>
<span class="w"> </span>TCP_CHECK<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>connect_timeout<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="c1"># 10秒无响应超时</span>
<span class="w"> </span>nb_get_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>delay_before_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>connect_port<span class="w"> </span><span class="m">80</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>real_server<span class="w"> </span><span class="m">192</span>.168.3.2<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>weight<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>TCP_CHECK<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>connect_timeout<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>delay_before_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>connect_port<span class="w"> </span><span class="m">80</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
</code></pre></div>
<p>配置备用调度器的keepalived,只需要将state MASTER 改为state BACKUP,降低priority 100 的值:</p>
<div class="highlight"><pre><span></span><code>global_defs<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>notification_email<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>cold_night@linuxzen.com
<span class="o">}</span>
<span class="w"> </span>notification_email_from<span class="w"> </span>root@linuxzen.com
<span class="w"> </span>smtp_server<span class="w"> </span>www.linuxzen.com
<span class="w"> </span>smtp_connect_timeout<span class="w"> </span><span class="m">30</span>
<span class="w"> </span>router_id<span class="w"> </span>LVS_DEVEL
<span class="o">}</span>
vrrp_instance<span class="w"> </span>VI_1<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>state<span class="w"> </span>BACKUP<span class="w"> </span><span class="c1"># 备机上改为BACKUP</span>
<span class="w"> </span>interface<span class="w"> </span>eth0
<span class="w"> </span>virtual_router_id<span class="w"> </span><span class="m">51</span><span class="w"> </span><span class="c1"># 主备的virtual_router_id的值必须相同</span>
<span class="w"> </span>priority<span class="w"> </span><span class="m">99</span><span class="w"> </span><span class="c1"># 备用优先级小于主调度器</span>
<span class="w"> </span>advert_int<span class="w"> </span><span class="m">1</span>
<span class="w"> </span>authentication<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>auth_type<span class="w"> </span>PASS
<span class="w"> </span>auth_pass<span class="w"> </span><span class="m">1111</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>virtual_ipaddress<span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="m">192</span>.168.3.253
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
virtual_server<span class="w"> </span><span class="m">192</span>.168.3.253<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>delay_loop<span class="w"> </span><span class="m">6</span>
<span class="w"> </span>lb_algo<span class="w"> </span>rr
<span class="w"> </span>lb_kind<span class="w"> </span>NAT
<span class="w"> </span>nat_mask<span class="w"> </span><span class="m">255</span>.255.255.0
<span class="w"> </span>persistence_timeout<span class="w"> </span><span class="m">50</span>
<span class="w"> </span>protocol<span class="w"> </span>TCP<span class="w"> </span>
<span class="w"> </span>real_server<span class="w"> </span><span class="m">192</span>.168.3.1<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>weight<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>TCP_CHECK<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>connect_timeout<span class="w"> </span><span class="m">10</span>
<span class="w"> </span>nb_get_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>delay_before_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>connect_port<span class="w"> </span><span class="m">80</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>real_server<span class="w"> </span><span class="m">192</span>.168.3.2<span class="w"> </span><span class="m">80</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span>weight<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>TCP_CHECK<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>connect_timeout<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>delay_before_retry<span class="w"> </span><span class="m">3</span>
<span class="w"> </span>connect_port<span class="w"> </span><span class="m">80</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
</code></pre></div>
<p>主备上启动keepalived:</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>keepalived<span class="w"> </span>start
</code></pre></div>
<h2 id="-">三、测试-----部署后端服务器</h2>
<p>在后端服务器安装nginx,这里仅部署一台然后创建3个基于ip的虚拟主机供测试:
绑定ip:</p>
<div class="highlight"><pre><span></span><code>ifconfig<span class="w"> </span>eth0:1<span class="w"> </span><span class="m">192</span>.168.3.4/24
ifconfig<span class="w"> </span>eth0:2<span class="w"> </span><span class="m">192</span>.168.3.5/24
ifconfig<span class="w"> </span>eth0:3<span class="w"> </span><span class="m">192</span>.168.3.6/24
</code></pre></div>
<p>安装nginx后编辑配置文件,在http块里添加:</p>
<div class="highlight"><pre><span></span><code>http<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.4:80<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span><span class="m">192</span>.168.3.4<span class="p">;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>html/s1<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.html<span class="w"> </span>index.htm<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.5:80<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span><span class="m">192</span>.168.3.5<span class="p">;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>html/s2<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.html<span class="w"> </span>index.htm<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>server<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>listen<span class="w"> </span><span class="m">192</span>.168.3.6:80<span class="p">;</span>
<span class="w"> </span>server_name<span class="w"> </span><span class="m">192</span>.168.3.5<span class="p">;</span>
<span class="w"> </span>location<span class="w"> </span>/<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>root<span class="w"> </span>html/s3<span class="p">;</span>
<span class="w"> </span>index<span class="w"> </span>index.html<span class="w"> </span>index.htm<span class="p">;</span>
<span class="w"> </span><span class="o">}</span>
<span class="w"> </span><span class="o">}</span>
<span class="o">}</span>
</code></pre></div>
<p>创建虚拟主机根目录,并创建不通的首页文档:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/local/nginx/html/
mkdir<span class="w"> </span>s1<span class="w"> </span>s2<span class="w"> </span>s3
<span class="nb">echo</span><span class="w"> </span>server1<span class="w"> </span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>s1/index.html
<span class="nb">echo</span><span class="w"> </span>server2<span class="w"> </span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>s2/index.html
<span class="nb">echo</span><span class="w"> </span>server3<span class="w"> </span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>s3/index.html
</code></pre></div>
<p>启动nginx:</p>
<div class="highlight"><pre><span></span><code>/usr/local/nginx/sbin/nginx
</code></pre></div>
<p>打开浏览器访问<code>http://192.168.3.253</code></p>
<p>刷新会看到显示不同的内容:server1,server2,server3(生产中的服务器应该是一样的)
[gallery link="file" order="DESC"]
现在停掉主调度器的keepalived</p>
<div class="highlight"><pre><span></span><code>pkill<span class="w"> </span>keepalived
</code></pre></div>
<p>查看备调度器的日志:</p>
<div class="highlight"><pre><span></span><code>cat<span class="w"> </span>/var/log/messages
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:27<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>VRRP_Instance<span class="o">(</span>VI_1<span class="o">)</span><span class="w"> </span>Transition<span class="w"> </span>to<span class="w"> </span>MASTER<span class="w"> </span>STATE
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:28<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>VRRP_Instance<span class="o">(</span>VI_1<span class="o">)</span><span class="w"> </span>Entering<span class="w"> </span>MASTER<span class="w"> </span>STATE
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:28<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>VRRP_Instance<span class="o">(</span>VI_1<span class="o">)</span><span class="w"> </span>setting<span class="w"> </span>protocol<span class="w"> </span>VIPs.
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:28<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>VRRP_Instance<span class="o">(</span>VI_1<span class="o">)</span><span class="w"> </span>Sending<span class="w"> </span>gratuitous<span class="w"> </span>ARPs<span class="w"> </span>on<span class="w"> </span>eth0<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.253
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:28<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>Netlink<span class="w"> </span>reflector<span class="w"> </span>reports<span class="w"> </span>IP<span class="w"> </span><span class="m">192</span>.168.3.253<span class="w"> </span>added
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:28<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_healthcheckers:<span class="w"> </span>Netlink<span class="w"> </span>reflector<span class="w"> </span>reports<span class="w"> </span>IP<span class="w"> </span><span class="m">192</span>.168.3.253<span class="w"> </span>added
Feb<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="m">16</span>:36:33<span class="w"> </span>cfhost<span class="w"> </span>Keepalived_vrrp:<span class="w"> </span>VRRP_Instance<span class="o">(</span>VI_1<span class="o">)</span><span class="w"> </span>Sending<span class="w"> </span>gratuitous<span class="w"> </span>ARPs<span class="w"> </span>on<span class="w"> </span>eth0<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="m">192</span>.168.3.253
</code></pre></div>
<p> </p>
<p>现在访问<code>http://192.168.3.253</code>依然可以访问.
大家也看到了备机keepalived只有检测主机的keepalived停止的时候才会切换vip,而不是检测一台real server的某一服务(比如检测80端口的HTTP)切换vip,所以在nginx进程停止的时候,如果服务器没有宕机这时候就无法实现故障转移,所以我们编写一个检测nginx状态的脚本结合keepalived实现故障转移:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1">#filename:nsc.sh</span>
ps<span class="w"> </span>aux<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>nginx<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span>grep<span class="w"> </span><span class="m">2</span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">1</span><span class="p">&</span>gt<span class="p">;&</span>amp<span class="p">;</span><span class="m">2</span><span class="w"> </span><span class="c1"># 过滤nginx进程</span>
<span class="k">if</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]]</span><span class="w"> </span><span class="c1"># 如果过滤有nginx进程会返回0则认为nginx存活</span>
<span class="k">then</span>
<span class="w"> </span>sleep<span class="w"> </span><span class="m">5</span><span class="w"> </span><span class="c1"># 使脚本进入休眠</span>
<span class="k">else</span>
<span class="c1"># 如果nginx没有存活尝试启动nginx,如果失败则杀死keepalived的进程</span>
<span class="w"> </span>/usr/local/nginx/sbin/nginx
<span class="w"> </span>ps<span class="w"> </span>aux<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>nginx<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>-v<span class="w"> </span>grep<span class="w"> </span><span class="m">2</span><span class="p">&</span>gt<span class="p">;</span><span class="w"> </span>/dev/null<span class="w"> </span><span class="m">1</span><span class="p">&</span>gt<span class="p">;&</span>amp<span class="p">;</span><span class="m">2</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="nv">$?</span><span class="w"> </span>-eq<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="o">]]</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span>pkill<span class="w"> </span>keepalived
<span class="w"> </span><span class="k">fi</span>
<span class="k">fi</span>
</code></pre></div>
<p>然后后台运行此脚本:</p>
<div class="highlight"><pre><span></span><code>nohup<span class="w"> </span>sh<span class="w"> </span>nsc.sh<span class="w"> </span><span class="p">&</span>
</code></pre></div>
<p>这样就实现了群集的高可靠和高可用.</p>MySQL优化笔记2012-02-04T13:36:00+08:002012-02-04T13:36:00+08:00coldtag:www.linuxzen.com,2012-02-04:/mysqlyou-hua-bi-ji.html<p>之前安装时没注意MySQL的优化,先想对MySQL做一下优化.首先看一下没有优化之前各个参数:
MySQL预编译参数:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span>./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/mysql<span class="w"> </span>--with-ssl<span class="w"> </span>--with-readline<span class="w"> </span>--with-big-tables<span class="w"> </span>--enable-assembler
</code></pre></div>
<p>top</p>
<div class="highlight"><pre><span></span><code><span class="m">32553</span><span class="w"> </span>mysql<span class="w"> </span><span class="m">20</span><span class="w"> </span><span class="m">0</span><span class="w"> </span>125m<span class="w"> </span>17m<span class="w"> </span><span class="m">4064</span><span class="w"> </span>S<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">1 …</span></code></pre></div><p>之前安装时没注意MySQL的优化,先想对MySQL做一下优化.首先看一下没有优化之前各个参数:
MySQL预编译参数:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span>./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/mysql<span class="w"> </span>--with-ssl<span class="w"> </span>--with-readline<span class="w"> </span>--with-big-tables<span class="w"> </span>--enable-assembler
</code></pre></div>
<p>top</p>
<div class="highlight"><pre><span></span><code><span class="m">32553</span><span class="w"> </span>mysql<span class="w"> </span><span class="m">20</span><span class="w"> </span><span class="m">0</span><span class="w"> </span>125m<span class="w"> </span>17m<span class="w"> </span><span class="m">4064</span><span class="w"> </span>S<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">1</span>.7<span class="w"> </span><span class="m">5</span>:13.01<span class="w"> </span>mysqld
</code></pre></div>
<p>使用mysqlreport获取MySQL运行参数:<!--more--></p>
<div class="highlight"><pre><span></span><code>MySQL<span class="w"> </span><span class="m">5</span>.0.40-log<span class="w"> </span>uptime<span class="w"> </span><span class="m">15</span><span class="w"> </span><span class="m">22</span>:1:21<span class="w"> </span>Sat<span class="w"> </span>Feb<span class="w"> </span><span class="m">4</span><span class="w"> </span><span class="m">10</span>:04:23<span class="w"> </span><span class="m">2012</span>
__<span class="w"> </span>Key<span class="w"> </span>_________________________________________________________________
Buffer<span class="w"> </span>used<span class="w"> </span><span class="m">62</span>.00k<span class="w"> </span>of<span class="w"> </span><span class="m">16</span>.00M<span class="w"> </span>%Used:<span class="w"> </span><span class="m">0</span>.38
<span class="w"> </span>Current<span class="w"> </span><span class="m">1</span>.90M<span class="w"> </span>%Usage:<span class="w"> </span><span class="m">11</span>.89
Write<span class="w"> </span>hit<span class="w"> </span><span class="m">22</span>.29%
Read<span class="w"> </span>hit<span class="w"> </span><span class="m">99</span>.83%
__<span class="w"> </span>Questions<span class="w"> </span>___________________________________________________________
Total<span class="w"> </span><span class="m">85</span>.58k<span class="w"> </span><span class="m">0</span>.1/s
<span class="w"> </span>DMS<span class="w"> </span><span class="m">77</span>.61k<span class="w"> </span><span class="m">0</span>.1/s<span class="w"> </span>%Total:<span class="w"> </span><span class="m">90</span>.69
<span class="w"> </span>Com_<span class="w"> </span><span class="m">5</span>.37k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">6</span>.28
<span class="w"> </span>COM_QUIT<span class="w"> </span><span class="m">2</span>.52k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">2</span>.95
<span class="w"> </span>+Unknown<span class="w"> </span><span class="m">78</span><span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">0</span>.09
Slow<span class="w"> </span><span class="m">10</span><span class="w"> </span>s<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s<span class="w"> </span><span class="m">0</span>.00<span class="w"> </span>%DMS:<span class="w"> </span><span class="m">0</span>.00<span class="w"> </span>Log:<span class="w"> </span>OFF
DMS<span class="w"> </span><span class="m">77</span>.61k<span class="w"> </span><span class="m">0</span>.1/s<span class="w"> </span><span class="m">90</span>.69
<span class="w"> </span>SELECT<span class="w"> </span><span class="m">72</span>.77k<span class="w"> </span><span class="m">0</span>.1/s<span class="w"> </span><span class="m">85</span>.03<span class="w"> </span><span class="m">93</span>.76
<span class="w"> </span>UPDATE<span class="w"> </span><span class="m">2</span>.68k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">3</span>.13<span class="w"> </span><span class="m">3</span>.45
<span class="w"> </span>INSERT<span class="w"> </span><span class="m">1</span>.09k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">1</span>.27<span class="w"> </span><span class="m">1</span>.41
<span class="w"> </span>DELETE<span class="w"> </span><span class="m">1</span>.07k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">1</span>.25<span class="w"> </span><span class="m">1</span>.38
<span class="w"> </span>REPLACE<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s<span class="w"> </span><span class="m">0</span>.00<span class="w"> </span><span class="m">0</span>.00
Com_<span class="w"> </span><span class="m">5</span>.37k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">6</span>.28
<span class="w"> </span>set_option<span class="w"> </span><span class="m">2</span>.60k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">3</span>.04
<span class="w"> </span>change_db<span class="w"> </span><span class="m">2</span>.52k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">2</span>.94
<span class="w"> </span>show_fields<span class="w"> </span><span class="m">77</span><span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">0</span>.09
__<span class="w"> </span>SELECT<span class="w"> </span>and<span class="w"> </span>Sort<span class="w"> </span>_____________________________________________________
Scan<span class="w"> </span><span class="m">5</span>.17k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span>%SELECT:<span class="w"> </span><span class="m">7</span>.10
Range<span class="w"> </span><span class="m">2</span>.75k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">3</span>.77
Full<span class="w"> </span>join<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s<span class="w"> </span><span class="m">0</span>.00
Range<span class="w"> </span>check<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s<span class="w"> </span><span class="m">0</span>.00
Full<span class="w"> </span>rng<span class="w"> </span>join<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s<span class="w"> </span><span class="m">0</span>.00
Sort<span class="w"> </span>scan<span class="w"> </span><span class="m">5</span>.97k<span class="w"> </span><span class="m">0</span>.0/s
Sort<span class="w"> </span>range<span class="w"> </span><span class="m">4</span>.30k<span class="w"> </span><span class="m">0</span>.0/s
Sort<span class="w"> </span>mrg<span class="w"> </span>pass<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
__<span class="w"> </span>Table<span class="w"> </span>Locks<span class="w"> </span>_________________________________________________________
Waited<span class="w"> </span><span class="m">24</span><span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span>%Total:<span class="w"> </span><span class="m">0</span>.03
Immediate<span class="w"> </span><span class="m">91</span>.00k<span class="w"> </span><span class="m">0</span>.1/s
__<span class="w"> </span>Tables<span class="w"> </span>______________________________________________________________
Open<span class="w"> </span><span class="m">36</span><span class="w"> </span>of<span class="w"> </span><span class="m">64</span><span class="w"> </span>%Cache:<span class="w"> </span><span class="m">56</span>.25
Opened<span class="w"> </span><span class="m">42</span><span class="w"> </span><span class="m">0</span>.0/s
__<span class="w"> </span>Connections<span class="w"> </span>_________________________________________________________
Max<span class="w"> </span>used<span class="w"> </span><span class="m">5</span><span class="w"> </span>of<span class="w"> </span><span class="m">100</span><span class="w"> </span>%Max:<span class="w"> </span><span class="m">5</span>.00
Total<span class="w"> </span><span class="m">2</span>.52k<span class="w"> </span><span class="m">0</span>.0/s
__<span class="w"> </span>Created<span class="w"> </span>Temp<span class="w"> </span>________________________________________________________
Disk<span class="w"> </span>table<span class="w"> </span><span class="m">4</span>.15k<span class="w"> </span><span class="m">0</span>.0/s
Table<span class="w"> </span><span class="m">7</span>.11k<span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span>Size:<span class="w"> </span><span class="m">32</span>.0M
File<span class="w"> </span><span class="m">5</span><span class="w"> </span><span class="m">0</span>.0/s
__<span class="w"> </span>Threads<span class="w"> </span>_____________________________________________________________
Running<span class="w"> </span><span class="m">1</span><span class="w"> </span>of<span class="w"> </span><span class="m">1</span>
Cached<span class="w"> </span><span class="m">0</span><span class="w"> </span>of<span class="w"> </span><span class="m">0</span><span class="w"> </span>%Hit:<span class="w"> </span><span class="m">0</span>.04
Created<span class="w"> </span><span class="m">2</span>.52k<span class="w"> </span><span class="m">0</span>.0/s
Slow<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
__<span class="w"> </span>Aborted<span class="w"> </span>_____________________________________________________________
Clients<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Connects<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
__<span class="w"> </span>Bytes<span class="w"> </span>_______________________________________________________________
Sent<span class="w"> </span><span class="m">226</span>.71M<span class="w"> </span><span class="m">164</span>.8/s
Received<span class="w"> </span><span class="m">12</span>.59M<span class="w"> </span><span class="m">9</span>.2/s
__<span class="w"> </span>InnoDB<span class="w"> </span>Buffer<span class="w"> </span>Pool<span class="w"> </span>__________________________________________________
Usage<span class="w"> </span><span class="m">304</span>.00k<span class="w"> </span>of<span class="w"> </span><span class="m">8</span>.00M<span class="w"> </span>%Used:<span class="w"> </span><span class="m">3</span>.71
Read<span class="w"> </span>hit<span class="w"> </span><span class="m">84</span>.42%
Pages
<span class="w"> </span>Free<span class="w"> </span><span class="m">493</span><span class="w"> </span>%Total:<span class="w"> </span><span class="m">96</span>.29
<span class="w"> </span>Data<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">3</span>.71<span class="w"> </span>%Drty:<span class="w"> </span><span class="m">0</span>.00
<span class="w"> </span>Misc<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>.00
<span class="w"> </span>Latched<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>.00
Reads<span class="w"> </span><span class="m">77</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>From<span class="w"> </span>file<span class="w"> </span><span class="m">12</span><span class="w"> </span><span class="m">0</span>.0/s<span class="w"> </span><span class="m">15</span>.58
<span class="w"> </span>Ahead<span class="w"> </span>Rnd<span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>Ahead<span class="w"> </span>Sql<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Writes<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Flushes<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Wait<span class="w"> </span>Free<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
__<span class="w"> </span>InnoDB<span class="w"> </span>Lock<span class="w"> </span>_________________________________________________________
Waits<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Current<span class="w"> </span><span class="m">0</span>
Time<span class="w"> </span>acquiring
<span class="w"> </span>Total<span class="w"> </span><span class="m">0</span><span class="w"> </span>ms
<span class="w"> </span>Average<span class="w"> </span><span class="m">0</span><span class="w"> </span>ms
<span class="w"> </span>Max<span class="w"> </span><span class="m">0</span><span class="w"> </span>ms
__<span class="w"> </span>InnoDB<span class="w"> </span>Data,<span class="w"> </span>Pages,<span class="w"> </span>Rows<span class="w"> </span>____________________________________________
Data
<span class="w"> </span>Reads<span class="w"> </span><span class="m">25</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>Writes<span class="w"> </span><span class="m">3</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>fsync<span class="w"> </span><span class="m">3</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>Pending
<span class="w"> </span>Reads<span class="w"> </span><span class="m">0</span>
<span class="w"> </span>Writes<span class="w"> </span><span class="m">0</span>
<span class="w"> </span>fsync<span class="w"> </span><span class="m">0</span>
Pages
<span class="w"> </span>Created<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
<span class="w"> </span>Read<span class="w"> </span><span class="m">19</span><span class="w"> </span><span class="m">0</span>.0/s
<span class="w"> </span>Written<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
Rows
<span class="w"> </span>Deleted<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
<span class="w"> </span>Inserted<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
<span class="w"> </span>Read<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
<span class="w"> </span>Updated<span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="m">0</span>/s
</code></pre></div>
<p>首先在预编译参数上进行优化</p>
<div class="highlight"><pre><span></span><code>./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/mysql<span class="w"> </span><span class="se">\</span>
--without-debug<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 取消调试模式提高性能</span>
--with-extra-charsets<span class="o">=</span>utf8,gbk<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 仅仅指定需要的默认字符集提高性能</span>
--enable-assembler<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 使用汇编模式提高性能</span>
--with-mysqld-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 以静态方式编译提高性能</span>
--with-client-ldflags<span class="o">=</span>-all-static<span class="w"> </span><span class="se">\</span>
--with-unix-socket-path<span class="o">=</span>/tmp/mysql.sock<span class="w"> </span><span class="se">\ </span><span class="w"> </span><span class="c1"># 使用unix socket提高性能</span>
--with-ssl
</code></pre></div>
<p>安装完成后进一步优化my.cnf:
因为MySQL 只会 Cache 索引(*.MYI),因此您只要将数据库中所有的 MYI 档案加总起来就是key buffer 的值,计算MYI档案的总大小:</p>
<div class="highlight"><pre><span></span><code>du<span class="w"> </span>-hc<span class="w"> </span><span class="sb">`</span>find<span class="w"> </span>/usr/local/mysql/var/<span class="w"> </span>-name<span class="w"> </span>*.MYI<span class="sb">`</span>
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_term_taxonomy.MYI
<span class="m">8</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_posts.MYI
<span class="m">8</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_usermeta.MYI
<span class="m">8</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_commentmeta.MYI
16K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_options.MYI
12K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_postmeta.MYI
<span class="m">8</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_comments.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_links.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_term_relationships.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_users.MYI
<span class="m">8</span>.0K<span class="w"> </span>/usr/local/mysql/var/myblog/wp_terms.MYI
16K<span class="w"> </span>/usr/local/mysql/var/mysql/help_relation.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/time_zone_name.MYI
16K<span class="w"> </span>/usr/local/mysql/var/mysql/help_keyword.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/func.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/time_zone.MYI
20K<span class="w"> </span>/usr/local/mysql/var/mysql/help_topic.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/columns_priv.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/procs_priv.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/time_zone_leap_second.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/user.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/tables_priv.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/host.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/time_zone_transition_type.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/proc.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/help_category.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/db.MYI
<span class="m">4</span>.0K<span class="w"> </span>/usr/local/mysql/var/mysql/time_zone_transition.MYI
192K<span class="w"> </span>total
</code></pre></div>
<p>修改my.cnf参数大小:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/my.cnf
<span class="c1"># 降低key_buffer的值</span>
<span class="nv">key_buffer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>4M
</code></pre></div>
<p>重启MySQL执行top命令:</p>
<div class="highlight"><pre><span></span><code><span class="m">18125</span><span class="w"> </span>mysql<span class="w"> </span><span class="m">20</span><span class="w"> </span><span class="m">0</span><span class="w"> </span>109m<span class="w"> </span>11m<span class="w"> </span><span class="m">2152</span><span class="w"> </span>S<span class="w"> </span><span class="m">0</span>.0<span class="w"> </span><span class="m">1</span>.1<span class="w"> </span><span class="m">0</span>:00.08<span class="w"> </span>mysqld
</code></pre></div>
<p>看到MySQL的内存利用率降低到1.1,这时候还不适宜执行mysqlreport查看等待启动一天后查看.由于现在访问量较低,所以参数适量调低,需要实时监控MySQL运行状况适当运行参数.</p>用pxe安装自己定制的发行版快速部署系统环境2012-02-02T11:37:00+08:002012-02-02T11:37:00+08:00coldtag:www.linuxzen.com,2012-02-02:/yong-pxean-zhuang-zi-ji-ding-zhi-de-fa-xing-ban-kuai-su-bu-shu-xi-tong-huan-jing.html<p>自己的发行版出来之后如何快速应用到企业中,可以使用PXE网络安装实现无人值守安装. 先解释下PXE安装,这是由Intel开发的一项技术,支持通过网络从远程服务器下载启动 …</p><p>自己的发行版出来之后如何快速应用到企业中,可以使用PXE网络安装实现无人值守安装. 先解释下PXE安装,这是由Intel开发的一项技术,支持通过网络从远程服务器下载启动文件系统本机,从而实现网络安装操作系统.</p>
<p>这里需要网络传输就需要ip,所以需要DHCP服务器,传输文件需要简单的传输协议就需要tftp服务器,安装的过程中需要传输安装文件就需要一个文件服务器(ftp/http/nfs等)
基本流程就是通过DHCP获取ip并获取下一步的服务器地址(也就是tftp),然后下载启动文件,通过启动文件获取指定的下一个服务器的地址(安装文件的远程路径)</p>
<p>使用环境:CentOS 5.5 32位</p>
<p>使用系统:根据CentOS定制的发行版ColdOS 0.1 32位</p>
<p>ip: 静态ip192.168.3.1</p>
<p>本文使用dhcpd分配ip,使用tftp-server提供下载启动文件,apache提供下载安装文件
1.安装配置tftp</p>
<p>安装tftp-server</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>tftp-server
</code></pre></div>
<p>配置tftp-server,编辑/etc/xinetd.d/tftp</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/etc/xinetd.d/tftp
</code></pre></div>
<p>修改配置文件</p>
<p> </p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>tftp
<span class="o">{</span>
<span class="w"> </span><span class="nv">socket_type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>dgram
<span class="w"> </span><span class="nv">protocol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>udp
<span class="w"> </span><span class="nb">wait</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>yes
<span class="w"> </span><span class="nv">user</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>root
<span class="w"> </span><span class="nv">server</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>/usr/sbin/in.tftpd
<span class="w"> </span><span class="nv">server_args</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>-s<span class="w"> </span>/tftpboot<span class="w"> </span><span class="c1"># 根目录</span>
<span class="w"> </span><span class="nv">disable</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>no<span class="w"> </span><span class="c1"># 把yes改为no启动tftp</span>
<span class="w"> </span><span class="nv">per_source</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">11</span>
<span class="w"> </span><span class="nv">cps</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">100</span><span class="w"> </span><span class="m">2</span>
<span class="w"> </span><span class="nv">flags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>IPv4
<span class="o">}</span>
</code></pre></div>
<p>启动tftp</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>xinetd<span class="w"> </span>restart
Stopping<span class="w"> </span>xinetd:<span class="w"> </span><span class="o">[</span><span class="w"> </span>OK<span class="w"> </span><span class="o">]</span>
Starting<span class="w"> </span>xinetd:<span class="w"> </span><span class="o">[</span><span class="w"> </span>OK<span class="w"> </span><span class="o">]</span>
</code></pre></div>
<p>2.在tftp根目录下创建启动文件供客户端下载:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 复制网卡启动文件</span>
cp<span class="w"> </span>/usr/lib/syslinux/pxelinux.0<span class="w"> </span>/tftpboot
</code></pre></div>
<p>将安装光盘上/image/pxeboot/initrd.img和vmlinux复制到/tftpboot/中</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 挂载光盘</span>
mount<span class="w"> </span>/dev/hdc<span class="w"> </span>/mnt
<span class="c1"># 复制文件</span>
cp<span class="w"> </span>/mnt/isolinux/vmlinuz<span class="w"> </span>/tftpboot/
cp<span class="w"> </span>/mnt/isolinux/initrd.img<span class="w"> </span>/tftpboot/
</code></pre></div>
<p>创建pxe启动菜单:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 创建存放启动菜单的目录</span>
<span class="nb">cd</span><span class="w"> </span>/tftpboot
mkdir<span class="w"> </span>pxelinux.cfg
<span class="c1"># 从安装光盘内复制启动菜单命名为default</span>
cp<span class="w"> </span>/mnt/isolinux/isolinux.cfg<span class="w"> </span>/tftpboot/pxelinux.cfg/default
</code></pre></div>
<p>编辑启动菜单:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/tftpboot/pxelinux.cfg/default
default<span class="w"> </span>ColdOS
prompt<span class="w"> </span><span class="m">1</span>
timeout<span class="w"> </span><span class="m">600</span>
label<span class="w"> </span>ColdOS
<span class="w"> </span>kernel<span class="w"> </span>vmlinuz
<span class="w"> </span>append<span class="w"> </span><span class="nv">ks</span><span class="o">=</span>http://192.168.3.1/cold/isolinux/ks.cfg<span class="w"> </span><span class="nv">initrd</span><span class="o">=</span>initrd.img<span class="w"> </span>text<span class="w"> </span><span class="c1"># 指定ks文件位置</span>
</code></pre></div>
<p>3.安装配置dhcp服务器
安装</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>dhcp
</code></pre></div>
<p>配置</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 创建配置文件</span>
cp<span class="w"> </span>/usr/share/doc/dhcp-3.0.5/dhcpd.conf.sample<span class="w"> </span>/etc/dhcpd.conf
<span class="c1"># 编辑配置文件</span>
vi<span class="w"> </span>/etc/dhcpd.conf
<span class="c1"># 修改配置文件:</span>
ddns-update-style<span class="w"> </span>interim<span class="p">;</span>
ignore<span class="w"> </span>client-updates<span class="p">;</span>
next-server<span class="w"> </span><span class="m">192</span>.168.3.1<span class="p">;</span><span class="w"> </span><span class="c1"># 指定tftp服务器</span>
filename<span class="w"> </span><span class="s2">"/pxelinux.0"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 指定启动文件</span>
subnet<span class="w"> </span><span class="m">192</span>.168.3.0<span class="w"> </span>netmask<span class="w"> </span><span class="m">255</span>.255.255.0<span class="w"> </span><span class="o">{</span>
<span class="w"> </span>option<span class="w"> </span>routers<span class="w"> </span><span class="m">192</span>.168.3.1<span class="p">;</span>
<span class="w"> </span>option<span class="w"> </span>subnet-mask<span class="w"> </span><span class="m">255</span>.255.255.0<span class="p">;</span>
<span class="w"> </span>option<span class="w"> </span>nis-domain<span class="w"> </span><span class="s2">"domain.org"</span><span class="p">;</span>
<span class="w"> </span>option<span class="w"> </span>domain-name<span class="w"> </span><span class="s2">"domain.org"</span><span class="p">;</span>
<span class="w"> </span>option<span class="w"> </span>domain-name-servers<span class="w"> </span><span class="m">192</span>.168.1.1<span class="p">;</span>
<span class="w"> </span>option<span class="w"> </span>time-offset<span class="w"> </span>-18000<span class="p">;</span><span class="w"> </span><span class="c1"># Eastern Standard Time</span>
<span class="w"> </span>range<span class="w"> </span>dynamic-bootp<span class="w"> </span><span class="m">192</span>.168.3.128<span class="w"> </span><span class="m">192</span>.168.3.254<span class="p">;</span>
<span class="w"> </span>default-lease-time<span class="w"> </span><span class="m">21600</span><span class="p">;</span>
<span class="w"> </span>max-lease-time<span class="w"> </span><span class="m">43200</span><span class="p">;</span>
<span class="o">}</span>
<span class="c1"># 启动dhcp服务器</span>
service<span class="w"> </span>dhcpd<span class="w"> </span>start
Starting<span class="w"> </span>dhcpd:<span class="w"> </span><span class="o">[</span><span class="w"> </span>OK<span class="w"> </span><span class="o">]</span>
</code></pre></div>
<p>4.安装配置http服务器
安装:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>httpd
</code></pre></div>
<p>首先把安装光盘里的内容拷到本地服务器上:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 创建目录</span>
mkdir<span class="w"> </span>-p<span class="w"> </span>/pxe/cold/
<span class="c1"># 复制文件</span>
<span class="o">(</span><span class="w"> </span><span class="nb">cd</span><span class="w"> </span>/mnt/<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>tar<span class="w"> </span>-cf<span class="w"> </span>-<span class="w"> </span>.<span class="w"> </span><span class="o">)</span><span class="w"> </span>¦<span class="w"> </span><span class="o">(</span><span class="nb">cd</span><span class="w"> </span>/pxe/cold<span class="w"> </span><span class="p">&</span>amp<span class="p">;&</span>amp<span class="p">;</span><span class="w"> </span>tar<span class="w"> </span>-xvf<span class="w"> </span>-<span class="w"> </span><span class="o">)</span>
</code></pre></div>
<p>配置apache</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 编辑配置文件:</span>
vi<span class="w"> </span>/etc/httpd/conf/httpd.conf
<span class="c1"># 更改apache根目录(将配置文件中的两处/var/www改成/pxe)</span>
DocumentRoot<span class="w"> </span><span class="s2">"/pxe"</span>
<span class="p">&</span>lt<span class="p">;</span>Directory<span class="w"> </span>/<span class="p">&</span>gt<span class="p">;</span>
<span class="w"> </span>Options<span class="w"> </span>FollowSymLinks
<span class="w"> </span>AllowOverride<span class="w"> </span>None
<span class="p">&</span>lt<span class="p">;</span>/Directory<span class="p">&</span>gt<span class="p">;</span>
<span class="p">&</span>lt<span class="p">;</span>Directory<span class="w"> </span><span class="s2">"/pxe"</span><span class="p">&</span>gt<span class="p">;</span>
<span class="w"> </span>Options<span class="w"> </span>Indexes<span class="w"> </span>FollowSymLinks
<span class="w"> </span>AllowOverride<span class="w"> </span>None
<span class="w"> </span>Order<span class="w"> </span>allow,deny
<span class="w"> </span>Allow<span class="w"> </span>from<span class="w"> </span>all
<span class="p">&</span>lt<span class="p">;</span>/Directory<span class="p">&</span>gt<span class="p">;</span>
</code></pre></div>
<p>配置完成后启动apache</p>
<div class="highlight"><pre><span></span><code>service<span class="w"> </span>httpd<span class="w"> </span>start
</code></pre></div>
<p>5.更改ks文件(ks文件可在CentOS图形界面下用kickstart程序直接生成,这里不做介绍)</p>
<p>因为网络安装,又需要部署cfengine的话,就需要更改cfengine的获取方式,首先要将文件夹打包:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/pxe/cold
tar<span class="w"> </span>-cvf<span class="w"> </span>cfengine.tar<span class="w"> </span>cfengine/
</code></pre></div>
<p>然后编辑ks文件</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 编辑ks文件</span>
vi<span class="w"> </span>/pxe/cold/isolinux/ks.cfg
install
cdrom
lang<span class="w"> </span>en_US.UTF-8
keyboard<span class="w"> </span>us
rootpw<span class="w"> </span>--iscrypted<span class="w"> </span><span class="nv">$1$OKNHES6P$tPdz9HxIp6</span>.QUvulqxNwa.
firewall<span class="w"> </span>--disable
authconfig<span class="w"> </span>--enableshadow<span class="w"> </span>--enablemd5
selinux<span class="w"> </span>--disabled
timezone<span class="w"> </span>--utc<span class="w"> </span>Asia/Chongqing
bootloader<span class="w"> </span>--location<span class="o">=</span>mbr<span class="w"> </span>--driveorder<span class="o">=</span>sda
url<span class="w"> </span>--url<span class="w"> </span>http://192.168.3.1/cold/<span class="w"> </span><span class="c1"># 添加指定安装文件的语句</span>
<span class="c1"># The following is the partition information you requested</span>
<span class="c1"># Note that any partitions you deleted are not expressed</span>
<span class="c1"># here so unless you clear all partitions first, this is</span>
<span class="c1"># not guaranteed to work</span>
clearpart<span class="w"> </span>--linux<span class="w"> </span>--drives<span class="o">=</span>sda
part<span class="w"> </span>/<span class="w"> </span>--fstype<span class="w"> </span>ext3<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow<span class="w"> </span>--maxsize<span class="o">=</span><span class="m">4096</span>
part<span class="w"> </span>/usr<span class="w"> </span>--fstype<span class="w"> </span>ext3<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow
part<span class="w"> </span>swap<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow<span class="w"> </span>--maxsize<span class="o">=</span><span class="m">512</span>
%packages
@core
@useful
@cfengine
%post<span class="w"> </span>--nochroot
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/masterfiles
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/inputs
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/outputs
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/bin
wget<span class="w"> </span>http://192.168.3.1/cold/cfengine.tar<span class="w"> </span><span class="c1"># 更改cfengine获取方式</span>
tar<span class="w"> </span>-xvf<span class="w"> </span>cfengine.tar
cp<span class="w"> </span>/cfengine/sbin/cf-*<span class="w"> </span>/mnt/sysimage/var/cfengine/bin/
cp<span class="w"> </span>-r<span class="w"> </span>/cfengine<span class="w"> </span>/mnt/sysimage/usr/local/
</code></pre></div>
<p>6.测试
开启另一台设备测试,机器启动时一般按F12即可进入网卡启动.</p>将cfengine集成到自己定制的发行版2012-01-19T11:35:00+08:002012-01-19T11:35:00+08:00coldtag:www.linuxzen.com,2012-01-19:/jiang-cfengineji-cheng-dao-zi-ji-ding-zhi-de-fa-xing-ban.html<p>上一篇文章介绍如何定制自己的发行版,并且完全根据CentOS进行定制,我们看到了用cfengine管理Linux系统的优势,如何快速部署cfengine呢,我们把cfengine集成到自己的定制的Linux中.
本文根据编译好的 …</p><p>上一篇文章介绍如何定制自己的发行版,并且完全根据CentOS进行定制,我们看到了用cfengine管理Linux系统的优势,如何快速部署cfengine呢,我们把cfengine集成到自己的定制的Linux中.
本文根据编译好的cfengine来进行定制,大家都知道编译之前会有很多依赖,所以我们需要安装系统的时候把依赖装好,然后把编译好的二进制文件复制到当前系统中.</p>
<p>本文完全根据<a title="根据CentOS定制自己的发行版" href="/post/36" target="_blank">上一篇</a>文章而来,挂载光驱安装工具等等这里就不介绍了.</p>
<!--more-->
<p>首先.记录cfengine依赖包的依赖关系,然后把这些包复制到定制发行版的目录下,cfengine所依赖的包如下:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 把依赖包添加到cfpkgs.list</span>
vi<span class="w"> </span>cfpkgs.list
<span class="c1"># 添加如下内容</span>
keyutils-libs-devel
zlib-devel
e2fsprogs-devel
libsepol-devel
libselinux-devel
db4-devel
pcre-devel
krb5-devel
flex
openssl-devel
<span class="c1"># 然后根据cfpkgs.list把包复制到发行版的CentOS目录下:</span>
<span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>cat<span class="w"> </span>cfpkgs.list<span class="w"> </span><span class="sb">`</span>:<span class="p">;</span><span class="w"> </span><span class="k">do</span><span class="w"> </span><span class="nv">file</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span><span class="w"> </span><span class="nv">$i</span><span class="w"> </span>¦<span class="w"> </span>sed<span class="w"> </span><span class="s1">'s/://g'</span><span class="sb">`</span><span class="p">;</span><span class="w"> </span>cp<span class="w"> </span>/mnt/CentOS/<span class="nv">$file</span>*<span class="w"> </span>/usr/cold/CentOS/<span class="p">;</span><span class="w"> </span><span class="k">done</span>
</code></pre></div>
<p>复制完包之后就需要编辑comps.xml,</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>/usr/cold/
<span class="c1"># 编辑comps.xml</span>
vi<span class="w"> </span>repodata/comps.xml
</code></pre></div>
<p>向comps.xml添加这几个包group节点</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><group>
<span class="w"> </span><id>cfengine</id>
<span class="w"> </span><name>Cfengine</name>
<span class="w"> </span><name<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Cfengine</name>
<span class="w"> </span><description>Cfengine<span class="w"> </span></description>
<span class="w"> </span><description<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Cfengine</description>
<span class="w"> </span><default>true</default>
<span class="w"> </span><uservisible>false</uservisible>
<span class="w"> </span><packagelist>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>keyutils-libs-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>zlib-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>e2fsprogs-devel<span class="w"> </span></packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>libsepol-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>libselinux-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>db4-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>pcre-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>krb5-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>flex</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>openssl-devel<span class="w"> </span></packagereq>
<span class="w"> </span></packagelist>
<span class="w"> </span></group>
</code></pre></div>
<p>然后通过命令创建源:</p>
<div class="highlight"><pre><span></span><code>createrepo<span class="w"> </span>-g<span class="w"> </span>repodata/comps.xml<span class="w"> </span>CentOS/
</code></pre></div>
<p>计算comps.xml的sha值</p>
<div class="highlight"><pre><span></span><code>sha1sum<span class="w"> </span>repodata/comps.xml
f6f086a3c2b7eee2050580aa3e74c841dd406dfc<span class="w"> </span>repodata/comps.xml
</code></pre></div>
<p>编辑repomd.xml</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>repodata/repomd.xml
</code></pre></div>
<p>将新的sha值更新到repomd.xml</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><data<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"group"</span>>
<span class="w"> </span><location<span class="w"> </span><span class="nv">href</span><span class="o">=</span><span class="s2">"repodata/comps.xml"</span>/>
<span class="w"> </span><checksum<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"sha"</span>>f6f086a3c2b7eee2050580aa3e74c841dd406dfc</checksum>
<span class="w"> </span><timestamp>1272586365</timestamp>
<span class="w"> </span></data>
</code></pre></div>
<p>接下来把我们编译好的cfengine(根据<a title="安装配置cfengine实现自动化配置Linux/Unix服务器" href="/post/17" target="_blank">安装配置cfengine实现自动化配置Linux/Unix服务器</a>)复制到定制的发行版的目录下:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>-r<span class="w"> </span>/usr/local/cfengine/<span class="w"> </span>/usr/cold/
</code></pre></div>
<p>然后编辑ks文件:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>isolinux/ks.cfg
<span class="c1"># 在末尾添加如下内容</span>
%packages
@core
@useful
@cfengine<span class="w"> </span><span class="c1"># 添加cfengine软件组</span>
<span class="c1"># %post是安装后执行的脚本,</span>
<span class="c1"># 还有一种标记 %pre是安装前执行的脚本,但不推荐使用这种脚本,</span>
<span class="c1"># 因为脚本出错会导致安装失败</span>
<span class="c1"># --nochroot是不切换根目录,这时候安装好的系统会挂载在 /mnt/sysimage下</span>
%post<span class="w"> </span>--nochroot
<span class="c1"># 创建cfengine的工作目录</span>
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/masterfiles
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/inputs
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/outputs
mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/sysimage/var/cfengine/bin
mkdir<span class="w"> </span>/a
mount<span class="w"> </span>/tmp/cdrom<span class="w"> </span>/a
<span class="c1"># 复制二进制文件和cfengine</span>
cp<span class="w"> </span>/a/cfengine/sbin/cf-*<span class="w"> </span>/mnt/sysimage/var/cfengine/bin/
cp<span class="w"> </span>-r<span class="w"> </span>/a/cfengine<span class="w"> </span>/mnt/sysimage/usr/local/
</code></pre></div>
<p>然后制作镜像:</p>
<div class="highlight"><pre><span></span><code>mkisofs<span class="w"> </span>-R<span class="w"> </span>-J<span class="w"> </span>-T<span class="w"> </span>-r<span class="w"> </span>-l<span class="w"> </span>-d<span class="w"> </span>-allow-multidot<span class="w"> </span>-allow-leading-dots<span class="w"> </span>-no-bak<span class="w"> </span>-o<span class="w"> </span>/usr/ColdOS-0.1-i386.iso<span class="w"> </span>-b<span class="w"> </span>isolinux/isolinux.bin<span class="w"> </span>-c<span class="w"> </span>isolinux/boot.cat<span class="w"> </span>-no-emul-boot<span class="w"> </span>-boot-load-size<span class="w"> </span><span class="m">4</span><span class="w"> </span>-boot-info-table<span class="w"> </span>.
</code></pre></div>
<p>然后把镜像下载下来进行安装,安装完成后执行</p>
<div class="highlight"><pre><span></span><code>/var/cfengine/bin/cf-key<span class="w"> </span><span class="c1"># 生成认证证书</span>
/var/cfengine/bin/cf-agent<span class="w"> </span>--bootstrap<span class="w"> </span>--policy-server<span class="w"> </span>cfhubip
/var/cfengine/bin/cf-agent<span class="w"> </span><span class="c1"># 执行承诺</span>
</code></pre></div>
<p>即可从cfhubip那里下载承诺文件执行承诺,根据cfhub进行本地配置
还可以根据%post更加自由的定制自己的系统,比如最小化安装后会无法使用本地镜像来使用yum可以定义一个局域网yum,在%post --nochroot添加如下内容:</p>
<div class="highlight"><pre><span></span><code>cat<span class="w"> </span><span class="s"><<__EOF__ > /mnt/sysimage/etc/yum.repos.d/CentOS-Media.repo</span>
<span class="s"># CentOS-Media.repo</span>
<span class="s"># yum --disablerepo=\* --enablerepo=cold [command]</span>
<span class="s">[cold]</span>
<span class="s">name=CentOS-$releasever - Media</span>
<span class="s">baseurl=http://192.168.0.254/CentOS</span>
<span class="s">gpgcheck=1</span>
<span class="s">enabled=1</span>
<span class="s">gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5</span>
<span class="s">__EOF__</span>
mv<span class="w"> </span>/mnt/sysimage/etc/yum.repos.d/CentOS-Base.repo<span class="w"> </span>/mnt/sysimage/etc/yum.repos.d/CentOS-Base.repo.bak
</code></pre></div>
<p>这里就不详细介绍,大家可根据这个更加自由的来定制自己的Linux发行版.</p>根据CentOS定制自己的发行版2012-01-18T11:13:00+08:002012-01-18T11:13:00+08:00coldtag:www.linuxzen.com,2012-01-18:/gen-ju-centosding-zhi-zi-ji-de-fa-xing-ban.html<p>本文使用的环境为CentOS5.5 32位.
首先要做的是最小化安装CentOS,就是安装过程中选择要安装包的时候使用自定义,然后去除所有 …</p><p>本文使用的环境为CentOS5.5 32位.
首先要做的是最小化安装CentOS,就是安装过程中选择要安装包的时候使用自定义,然后去除所有要安装的包,系统就会默认最小化安装系统.
装完系统会再root的根目录下生成3个文件,我们用到两个:
anaconda-ks.cfg : kisckstart脚本,记录安装过程的配置,包括选择语言,选择键盘,分区,root密码等等等等
install.log : 记录安装过程中所安装的包
我们首先要做的就是建立我们的自己发行版的目录,在一个剩余大小大于4G的分区创建自己发行版的目录,比如我的发行版叫ColdOS,然后挂载DVD光盘,把光盘上的内容复制到自己发行版的目录:<!--more--></p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>/usr/cold<span class="w"> </span><span class="c1"># 创建发行版目录</span>
mount<span class="w"> </span>/dev/hdc<span class="w"> </span>/mnt<span class="w"> </span><span class="c1"># 挂载IDE光驱</span>
<span class="c1"># 或</span>
mount<span class="w"> </span>/dev/cdrom<span class="w"> </span>/mnt
<span class="c1"># 如挂载本地ISO镜像使用下面命令</span>
mount<span class="w"> </span>-o<span class="w"> </span>loop<span class="w"> </span>iso路径<span class="w"> </span>/mnt
<span class="c1"># 然后使用 tar命令把光盘的内容复制到/usr/cold</span>
<span class="c1"># 都说这个命令比cp快,在256内存的虚拟机测试也就快了几十秒,个人觉得最大的用处</span>
<span class="c1"># 就是解决了cp无法复制隐藏文件的问题</span>
<span class="o">(</span><span class="w"> </span><span class="nb">cd</span><span class="w"> </span>/mnt/<span class="w"> </span><span class="o">&&</span><span class="w"> </span>tar<span class="w"> </span>-cf<span class="w"> </span>-<span class="w"> </span>.<span class="w"> </span><span class="o">)</span><span class="w"> </span>¦<span class="w"> </span><span class="o">(</span><span class="nb">cd</span><span class="w"> </span>/usr/cold<span class="w"> </span><span class="o">&&</span><span class="w"> </span>tar<span class="w"> </span>-xvfp<span class="w"> </span>-<span class="w"> </span><span class="o">)</span><span class="w"> </span><span class="c1"># tar -p选项是保留原有权限</span>
</code></pre></div>
<p>复制完成后发行版目录应该是:</p>
<div class="highlight"><pre><span></span><code>ls<span class="w"> </span>-la<span class="w"> </span>/usr/cold
total<span class="w"> </span><span class="m">511</span>
drwxr-xr-x<span class="w"> </span><span class="m">7</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">6144</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>.
drwxr-xr-x<span class="w"> </span><span class="m">21</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">18</span>:07<span class="w"> </span>..
drwxrwxr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span><span class="m">501</span><span class="w"> </span><span class="m">501</span><span class="w"> </span><span class="m">421888</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>CentOS
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">112</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>.discinfo
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">212</span><span class="w"> </span>Jun<span class="w"> </span><span class="m">15</span><span class="w"> </span><span class="m">2008</span><span class="w"> </span>EULA
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">18009</span><span class="w"> </span>Jun<span class="w"> </span><span class="m">15</span><span class="w"> </span><span class="m">2008</span><span class="w"> </span>GPL
drwxr-xr-x<span class="w"> </span><span class="m">4</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">2048</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>images
drwxr-xr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">2048</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>isolinux
drwxr-xr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">22528</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>NOTES
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">655</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-cs
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1401</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-cs.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">839</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-de
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1571</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-de.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">694</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-en
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1367</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-en.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">694</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-en_US
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1367</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-en_US.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">788</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-es
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1619</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-es.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">852</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-fr
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1641</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-fr.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">766</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-ja
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1565</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-ja.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">706</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-nl
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1433</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-nl.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">752</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-pt_BR
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1480</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-pt_BR.html
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">801</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-ro
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1473</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">27</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>RELEASE-NOTES-ro.html
drwxr-xr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">2048</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>repodata
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1512</span><span class="w"> </span>Jun<span class="w"> </span><span class="m">15</span><span class="w"> </span><span class="m">2008</span><span class="w"> </span>RPM-GPG-KEY-beta
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">1504</span><span class="w"> </span>Jun<span class="w"> </span><span class="m">15</span><span class="w"> </span><span class="m">2008</span><span class="w"> </span>RPM-GPG-KEY-CentOS-5
-r--r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">7048</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>TRANS.TBL
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">413</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>.treeinfo
</code></pre></div>
<p>现在对系统进行精简:</p>
<div class="highlight"><pre><span></span><code>rm<span class="w"> </span>-f<span class="w"> </span>RELEASE-NOTES-*
<span class="w"> </span>rm<span class="w"> </span>-rf<span class="w"> </span>NOTES/
rm<span class="w"> </span>-f<span class="w"> </span>RPM-GPG-KEY-*
rm<span class="w"> </span>-f<span class="w"> </span>EULA
rm<span class="w"> </span>-f<span class="w"> </span>GPL
rm<span class="w"> </span>-f<span class="w"> </span>CentOS/*<span class="w"> </span><span class="c1"># 删除所有rpm包,等会根据install.log复制过来,保证系统最小化</span>
</code></pre></div>
<p>精简完后应该是这样子的:</p>
<div class="highlight"><pre><span></span><code>s<span class="w"> </span>-la<span class="w"> </span>/usr/cold/
total<span class="w"> </span><span class="m">20</span>
drwxr-xr-x<span class="w"> </span><span class="m">6</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">19</span>:39<span class="w"> </span>.
drwxr-xr-x<span class="w"> </span><span class="m">16</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">23</span>:14<span class="w"> </span>..
drwxrwxr-x<span class="w"> </span><span class="m">3</span><span class="w"> </span><span class="m">501</span><span class="w"> </span><span class="m">501</span><span class="w"> </span><span class="m">172032</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">14</span><span class="w"> </span><span class="m">02</span>:23<span class="w"> </span>CentOS
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">112</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>.discinfo
drwxr-xr-x<span class="w"> </span><span class="m">4</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>images
drwxr-xr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">14</span><span class="w"> </span><span class="m">00</span>:36<span class="w"> </span>isolinux
drwxr-xr-x<span class="w"> </span><span class="m">2</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">4096</span><span class="w"> </span>Dec<span class="w"> </span><span class="m">14</span><span class="w"> </span><span class="m">01</span>:42<span class="w"> </span>repodata
-r--r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">7048</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>TRANS.TBL
-rw-r--r--<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">413</span><span class="w"> </span>Apr<span class="w"> </span><span class="m">30</span><span class="w"> </span><span class="m">2010</span><span class="w"> </span>.treeinfo
</code></pre></div>
<p>现在根据install.log创建packages.list,用来安装所需要的rpm包:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 根据install.log,提取其中的rpm包名,</span>
cat<span class="w"> </span>install.log<span class="w"> </span>¦<span class="w"> </span>grep<span class="w"> </span>Installing<span class="w"> </span>¦<span class="w"> </span>awk<span class="w"> </span><span class="s1">'{print $2}'</span><span class="w"> </span>¦<span class="w"> </span>awk<span class="w"> </span>-F<span class="s1">':'</span><span class="w"> </span><span class="s1">'{ if (NF==2){ print $2} else {print $1}}'</span><span class="w"> </span>><span class="w"> </span>packages.list
<span class="c1"># 如果仅仅最小化安装就失去了定制自己的发行版的意义</span>
<span class="c1"># 向packages.list添加几个常用的工具,需要先用yum安装一遍记住包的依赖关系</span>
<span class="c1"># 把依赖关系的包也放入packages.list</span>
<span class="c1"># 比如man依赖bzip2 groff</span>
vi<span class="w"> </span>packages.list
<span class="c1"># 末尾添加如下内容:</span>
setuptool
lszrz<span class="w"> </span>wget
kernel-devel
kernel-headers
libgomp
cpp
glibc-headers
glibc-devel
gcc
make
which
bzip2
groff
man
<span class="c1"># 根据packages.list进行复制</span>
<span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>cat<span class="w"> </span>packages.list<span class="w"> </span><span class="sb">`</span><span class="p">;</span><span class="k">do</span><span class="w"> </span>cp<span class="w"> </span>-p<span class="w"> </span>-f<span class="w"> </span>/mnt/CentOS/<span class="s2">"</span><span class="nv">$i</span><span class="s2">"</span>*<span class="w"> </span>/usr/cold/CentOS/<span class="p">;</span><span class="k">done</span>
</code></pre></div>
<p>复制完所需的RPM包之后我们如何来让系统安装的时候安装我们自定义添加的包呢?
首先我们需要编辑repodata/comps.xml,但是comps.xml文件内容太杂,大概 2万多行,所以我们需要对comps.xml进行一个预处理:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># comps.xml包含最多的是各国语言</span>
<span class="c1"># 我们先去除不需要的语言,这里我只需要英文,</span>
sed<span class="w"> </span>-ri<span class="w"> </span><span class="s1">'/xml:lang/ {/en_GB/!d}'</span><span class="w"> </span>comps.xml<span class="w"> </span><span class="c1"># 如果需要保留其他语言比如中文,在en_GB后添加"¦zh_CN"</span>
</code></pre></div>
<p>去除了各国语言的comps.xml大概包含2000多行,现在我们要进一步处理,删除除了id为core的group的其他节点,删除完后comps.xml大概只剩下73行.现在我们要把我们自己添加的包顶一个group节点,在紧跟id为core的group节点(也就是<code></group></code>后面)添加如下内容:</p>
<div class="highlight"><pre><span></span><code><group>
<span class="w"> </span><id>useful</id>
<span class="w"> </span><name>Useful</name>
<span class="w"> </span><name<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Useful</name>
<span class="w"> </span><description>Useful<span class="w"> </span>tools<span class="w"> </span><span class="k">for</span><span class="w"> </span>administartor<span class="w"> </span></description>
<span class="w"> </span><description<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Useful<span class="w"> </span>tools<span class="w"> </span><span class="k">for</span><span class="w"> </span>administartor</description>
<span class="w"> </span><default>true</default>
<span class="w"> </span><uservisible>false</uservisible>
<span class="w"> </span><packagelist>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>setuptool</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>lszrz</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>wget</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>kernel-headers</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>libgomp</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>cpp</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>glibc-headers</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>glibc-devel</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>gcc</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>make</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>which</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>bzip2</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>groff</packagereq>
<span class="w"> </span><packagereq<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"default"</span>>man</packagereq>
<span class="w"> </span></packagelist>
<span class="w"> </span></group>
</code></pre></div>
<p>上面添加了一个id为useful的group节点,下面把这两个节点放到一个类别里:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><category>
<span class="w"> </span><id>cold</id>
<span class="w"> </span><name>Cold</name>
<span class="w"> </span><name<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Cold</name>
<span class="w"> </span><description>Cold<span class="w"> </span>Linux</description>
<span class="w"> </span><description<span class="w"> </span>xml:lang<span class="o">=</span><span class="s2">"en_GB"</span>>Cold<span class="w"> </span>Linux</description>
<span class="w"> </span><display_order>92</display_order>
<span class="w"> </span><grouplist>
<span class="w"> </span><groupid>core</groupid>
<span class="w"> </span><groupid>useful</groupid>
<span class="w"> </span></grouplist>
<span class="w"> </span></category>
</code></pre></div>
<p>然后根据我们的comps.xml创建源:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 安装所需要的工具</span>
yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>createrepo<span class="w"> </span>anaconda<span class="w"> </span>anaconda-runtime
<span class="c1"># 创建源</span>
createrepo<span class="w"> </span>-g<span class="w"> </span>/usr/cold/repodata/comps.xml<span class="w"> </span>/usr/cold/CentOS
<span class="c1"># 完成后会有如下提示:</span>
<span class="m">224</span>/224<span class="w"> </span>-<span class="w"> </span>kudzu-1.2.57.1.24-1.el5.centos.i386.rpm
Saving<span class="w"> </span>Primary<span class="w"> </span>metadata
Saving<span class="w"> </span>file<span class="w"> </span>lists<span class="w"> </span>metadata
Saving<span class="w"> </span>other<span class="w"> </span>metadata
</code></pre></div>
<p>由于我们编辑了comps.xml,所以comps.xml的sha值会改变,这样就会导致跟repomd.xml中所记录的不同,安装的时候会报错:An error occurred umounting the CD. Please make sure you'are not accessing /mnt/source from the shell on tty2 an the click OK retry.
所以我们更改完comps.xml要计算comps.xml的sha值</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 计算comps.xml的sha值</span>
sha1sum<span class="w"> </span>repodata/comps.xml
c1d304cae50f969370a72d95e3cd2f71087fc73a<span class="w"> </span>repodata/comps.xml
</code></pre></div>
<p>然后更新到repomd.xml中编辑repodata/repomd.xml找到location href="repodata/comps.xml"/的一个data节点把sha值更新为刚刚计算的</p>
<div class="highlight"><pre><span></span><code><data<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"group"</span>>
<span class="w"> </span><location<span class="w"> </span><span class="nv">href</span><span class="o">=</span><span class="s2">"repodata/comps.xml"</span>/>
<span class="w"> </span><checksum<span class="w"> </span><span class="nv">type</span><span class="o">=</span><span class="s2">"sha"</span>>c1d304cae50f969370a72d95e3cd2f71087fc73a</checksum>
<span class="w"> </span><timestamp>1272586365</timestamp>
<span class="w"> </span></data>
</code></pre></div>
<p>comps.xml里新加了一个咱们的useful 组,怎么使系统安装我们定义的包呢?怎么自定义安装过程呢?下面将讲解根据anaconda-ks.cfg文件定义安装过程:
首先复制anaconda-ks.cfg到我们的发行版目录:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>~/anaconda-ks.cfg<span class="w"> </span>/usr/cold/isolinux/ks.cfg<span class="w"> </span><span class="c1"># 复制到isolinux下并命名为ks.cfg</span>
<span class="nb">cd</span><span class="w"> </span>/usr/cold
<span class="c1"># 编辑kickstart脚本</span>
vi<span class="w"> </span>isolinux/ks.cfg
install<span class="w"> </span><span class="c1"># 定义安装</span>
cdrom<span class="w"> </span><span class="c1"># 从光盘安装</span>
lang<span class="w"> </span>en_US.UTF-8<span class="w"> </span><span class="c1"># 安装语言为英文</span>
keyboard<span class="w"> </span>us<span class="w"> </span><span class="c1"># 定义键盘布局</span>
rootpw<span class="w"> </span>--iscrypted<span class="w"> </span><span class="nv">$1$OKNHES6P$tPdz9HxIp6</span>.QUvulqxNwa.<span class="w"> </span><span class="c1"># 定义root密码(你安装的时候提供的密码)</span>
firewall<span class="w"> </span>--disable<span class="w"> </span><span class="c1"># 禁用防火墙</span>
authconfig<span class="w"> </span>--enableshadow<span class="w"> </span>--enablemd5<span class="w"> </span><span class="c1"># 使用md5加密</span>
selinux<span class="w"> </span>--disabled<span class="w"> </span><span class="c1"># 禁用selinux</span>
timezone<span class="w"> </span>--utc<span class="w"> </span>Asia/Chongqing<span class="w"> </span><span class="c1"># 定义时区</span>
bootloader<span class="w"> </span>--location<span class="o">=</span>mbr<span class="w"> </span>--driveorder<span class="o">=</span>sda<span class="w"> </span><span class="c1"># 在一块硬盘上安装mbr</span>
<span class="c1"># The following is the partition information you requested</span>
<span class="c1"># Note that any partitions you deleted are not expressed</span>
<span class="c1"># here so unless you clear all partitions first, this is</span>
<span class="c1"># not guaranteed to work</span>
<span class="c1"># 如果想安装过程中手动分区就把下面几行注释掉</span>
clearpart<span class="w"> </span>--linux<span class="w"> </span>--drives<span class="o">=</span>sda<span class="w"> </span><span class="c1"># 格式化sda</span>
part<span class="w"> </span>/<span class="w"> </span>--fstype<span class="w"> </span>ext3<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow<span class="w"> </span>--maxsize<span class="o">=</span><span class="m">4096</span><span class="w"> </span><span class="c1"># 创建/分区大小为4个G</span>
part<span class="w"> </span>/usr<span class="w"> </span>--fstype<span class="w"> </span>ext3<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow<span class="w"> </span><span class="c1"># 创建/usr分区,大小为剩余空间</span>
part<span class="w"> </span>swap<span class="w"> </span>--size<span class="o">=</span><span class="m">1</span><span class="w"> </span>--grow<span class="w"> </span>--maxsize<span class="o">=</span><span class="m">512</span><span class="w"> </span><span class="c1"># 创建swap大小为512M</span>
%packages<span class="w"> </span><span class="c1"># 定义安装时安装的包</span>
@core<span class="w"> </span><span class="c1"># 最小化安装的包</span>
@useful<span class="w"> </span><span class="c1"># 自定义的包</span>
</code></pre></div>
<p>然后就要修改配置文件使安装时使用ks.cfg的配置来安装,修改isolinux.cfg:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>isolinux/isolinux.cfg
default<span class="w"> </span>linux<span class="w"> </span><span class="c1"># 默认启动的label</span>
prompt<span class="w"> </span><span class="m">1</span>
timeout<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="c1"># 等待时间为1秒钟</span>
display<span class="w"> </span>boot.msg
F1<span class="w"> </span>boot.msg
F2<span class="w"> </span>options.msg
F3<span class="w"> </span>general.msg
F4<span class="w"> </span>param.msg
F5<span class="w"> </span>rescue.msg
label<span class="w"> </span>linux
<span class="w"> </span>kernel<span class="w"> </span>vmlinuz
<span class="w"> </span>append<span class="w"> </span><span class="nv">ks</span><span class="o">=</span>cdrom:/isolinux/ks.cfg<span class="w"> </span><span class="nv">initrd</span><span class="o">=</span>initrd.img<span class="w"> </span>text<span class="w"> </span><span class="c1">#修改默认linux的label,ks使用自定义的ks,并文本启动</span>
label<span class="w"> </span>text
<span class="w"> </span>kernel<span class="w"> </span>vmlinuz
<span class="w"> </span>append<span class="w"> </span><span class="nv">initrd</span><span class="o">=</span>initrd.img<span class="w"> </span>text
label<span class="w"> </span>ks
<span class="w"> </span>kernel<span class="w"> </span>vmlinuz
<span class="w"> </span>append<span class="w"> </span>ks<span class="w"> </span><span class="nv">initrd</span><span class="o">=</span>initrd.img
label<span class="w"> </span><span class="nb">local</span>
<span class="w"> </span>localboot<span class="w"> </span><span class="m">1</span>
label<span class="w"> </span>memtest86
<span class="w"> </span>kernel<span class="w"> </span>memtest
<span class="w"> </span>append<span class="w"> </span>-
</code></pre></div>
<p>到这里配置就基本完成,下面就是制作iso镜像进行安装测试:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 首先安装工具</span>
yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>mkisofs
<span class="c1"># 创建iso镜像:</span>
<span class="nb">cd</span><span class="w"> </span>/usr/cold
mkisofs<span class="w"> </span>-R<span class="w"> </span>-J<span class="w"> </span>-T<span class="w"> </span>-r<span class="w"> </span>-l<span class="w"> </span>-d<span class="w"> </span>-allow-multidot<span class="w"> </span>-allow-leading-dots<span class="w"> </span>-no-bak<span class="w"> </span>-o<span class="w"> </span>/usr/ColdOS-0.1-i386.iso<span class="w"> </span>-b<span class="w"> </span>isolinux/isolinux.bin<span class="w"> </span>-c<span class="w"> </span>isolinux/boot.cat<span class="w"> </span>-no-emul-boot<span class="w"> </span>-boot-load-size<span class="w"> </span><span class="m">4</span><span class="w"> </span>-boot-info-table<span class="w"> </span>.
</code></pre></div>
<p>然后就可以把镜像下载下来进行安装测试了...下一篇就会介绍如何把之前一篇文章介绍的cfengine集成到自己的发行版</p>安装配置cfengine实现自动化配置Linux/Unix服务器2012-01-17T23:06:00+08:002012-01-17T23:06:00+08:00coldtag:www.linuxzen.com,2012-01-17:/an-zhuang-pei-zhi-cfengineshi-xian-zi-dong-hua-pei-zhi-linuxunixfu-wu-qi.html<p>cfengine(配置引擎)是一种 UNIX 管理工具,其目的是使简单的管理的任务自动化,使困难的任务变得较容易。Cfengine 适用于管理各种环境 …</p><p>cfengine(配置引擎)是一种 UNIX 管理工具,其目的是使简单的管理的任务自动化,使困难的任务变得较容易。Cfengine 适用于管理各种环境,从一台主机到上万台主机的机群均可使用. cfengine的版本2和版本3存在很大差异,这里使用最新版本的cfengine3:</p>
<p>cfengine执行和分发的策略被称为承诺,cfengine建议的结构是:</p>
<ul>
<li>一个版本控制器(subversion)用来创建策略</li>
<li>多个策略分发器(cfenginehub)用来分发策略----server</li>
<li>多个策略执行器(cfengine host)用来执行策略-----client,</li>
</ul>
<p>本文使用的环境为:</p>
<ul>
<li>cfhub: 172.16.1.1/24</li>
<li>cfhost: 172.16.1.2/24</li>
</ul>
<p>系统为CentOS 5.5 32位最小化安装,本文将不会解释一些基本命令,如有疑问请移步google.</p>
<p>下载:</p>
<div class="highlight"><pre><span></span><code>wget<span class="w"> </span>https://cfengine.com/source-code/download?file<span class="o">=</span>cfengine-3.2.3.tar.gz
</code></pre></div>
<p>安装依赖:</p>
<div class="highlight"><pre><span></span><code>yum<span class="w"> </span>-y<span class="w"> </span>install<span class="w"> </span>db4-devel<span class="w"> </span>pcre-devel<span class="w"> </span>openssl-devel<span class="w"> </span>flex
</code></pre></div>
<p>编译安装:</p>
<div class="highlight"><pre><span></span><code>tar<span class="w"> </span>-zxvf<span class="w"> </span>download<span class="se">\?</span>file<span class="se">\=</span>cfengine-3.2.3.tar.gz
<span class="nb">cd</span><span class="w"> </span>cfengine-3.2.3/
./configure<span class="w"> </span>--prefix<span class="o">=</span>/usr/local/cfengine
make<span class="w"> </span><span class="o">&&</span><span class="w"> </span>make<span class="w"> </span>install
</code></pre></div>
<p>为了保证cfengine正常工作创建cfengine工作目录:</p>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>-p<span class="w"> </span>/var/cfengine/masterfiles<span class="w"> </span><span class="c1"># 存放要分发的承诺</span>
mkdir<span class="w"> </span>/var/cfengine/inputs<span class="w"> </span><span class="c1"># 存放要执行的承诺</span>
mkdir<span class="w"> </span>/var/cfengine/outputs<span class="w"> </span><span class="c1"># 存放执行承诺的输出</span>
mkdir<span class="w"> </span>/var/cfengine/bin<span class="w"> </span><span class="c1"># 存放二进制文件</span>
</code></pre></div>
<p>复制二进制文件:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>/usr/local/cfengine/sbin/cf-*<span class="w"> </span>/var/cfengine/bin/
ls<span class="w"> </span>-l<span class="w"> </span>/var/cfengine/bin/
total<span class="w"> </span><span class="m">964</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">235958</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-agent<span class="w"> </span><span class="c1"># 执行承诺</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">63095</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-execd<span class="w"> </span><span class="c1"># 用于替代cron的程序,定时执行cf-agent</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">39076</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-key<span class="w"> </span><span class="c1"># 生成用于认证的证书</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">78711</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-know<span class="w"> </span><span class="c1"># 从大量承诺(知识建模代理)生成一个 ISO 标准的 Topic Map 的命令</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">150846</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-monitord<span class="w"> </span><span class="c1"># 负责收集有关系统状态信息的守护进程</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">16078</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-promises<span class="w"> </span><span class="c1"># 检查承诺语法的程序</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">107349</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-report<span class="w"> </span><span class="c1"># 生成报告</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">44160</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-runagent<span class="w"> </span><span class="c1"># 用来执行远程的cf-agent</span>
-rwxr-xr-x<span class="w"> </span><span class="m">1</span><span class="w"> </span>root<span class="w"> </span>root<span class="w"> </span><span class="m">172251</span><span class="w"> </span>Jan<span class="w"> </span><span class="m">13</span><span class="w"> </span><span class="m">2012</span><span class="w"> </span>cf-serverd<span class="w"> </span><span class="c1"># 分发承诺的守护进程</span>
</code></pre></div>
<p>一个cfengine的hello, world:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/var/cfengine/inputs/test.cf<span class="w"> </span><span class="c1"># 新建一个承诺文件,添加下面内容</span>
<span class="c1">###################</span>
<span class="c1"># 这里顺便解释一下cfengine的语法,cfengine的语法大都如下</span>
<span class="c1"># <它是什么> <它对什么起作用> <它叫什么></span>
body<span class="w"> </span>common<span class="w"> </span>control<span class="w"> </span><span class="c1"># 一个body 对common组启作用,名字是control(名字为control的common的组是最重要的一个组,cfengine以这个组为起点</span>
<span class="o">{</span>
<span class="nv">bundlesequence</span><span class="w"> </span><span class="o">=</span>><span class="w"> </span><span class="o">{</span><span class="w"> </span><span class="s2">"test"</span><span class="w"> </span><span class="o">}</span><span class="p">;</span><span class="w"> </span><span class="c1"># 定义要执行的承诺束为test</span>
<span class="o">}</span>
bundle<span class="w"> </span>agent<span class="w"> </span><span class="nb">test</span><span class="w"> </span><span class="c1"># 一个承诺束,对cf-agent起作用,叫test</span>
<span class="o">{</span>
reports:<span class="w"> </span><span class="c1"># 一个报告(仅仅显示消息,不对系统做改变)</span>
<span class="w"> </span>cfengine_3::<span class="w"> </span><span class="c1"># 一个类,对cfengine 版本3的策略执行点起作用(cfengine用这个来替代程序语言里的if-else)</span>
<span class="w"> </span><span class="s2">"Hello world!"</span><span class="p">;</span><span class="w"> </span><span class="c1"># 承诺内容,显示"hello world!"</span>
<span class="o">}</span>
/var/cfengine/bin/cf-promises<span class="w"> </span>-f<span class="w"> </span>/var/cfengine/inputs/test.cf<span class="w"> </span><span class="c1"># 检查承诺语法</span>
/var/cfengine/bin/cf-agent<span class="w"> </span>-f<span class="w"> </span>/var/cfengine/inputs/test.cf<span class="w"> </span><span class="c1"># 执行承诺</span>
R:<span class="w"> </span>Hello<span class="w"> </span>world!<span class="w"> </span><span class="c1"># R:代表一个report.</span>
</code></pre></div>
<p>为策略分发点创建配置文件:</p>
<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>/usr/local/cfengine/share/cfengine/masterfiles/*<span class="w"> </span>/var/cfengine/masterfile
</code></pre></div>
<p>编辑策略文件:
新建一个cftest1.cf承诺文件</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/var/cfengine/masterfiles/cftest1.cf<span class="w"> </span><span class="c1"># 新建一个cftest1.cf</span>
<span class="c1"># 添加如下内容:</span>
bundle<span class="w"> </span>agent<span class="w"> </span><span class="nb">test</span>
<span class="o">{</span>
reports:
<span class="w"> </span>cfengine_3::
<span class="w"> </span><span class="s2">"I'am cfengine 3 client"</span><span class="p">;</span>
<span class="o">}</span>
</code></pre></div>
<p>然后编辑promises.cf文件加入执行这个承诺束的支持:</p>
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/var/cfengine/masterfiles/promises.cf
body<span class="w"> </span>common<span class="w"> </span>control
<span class="o">{</span>
<span class="w"> </span><span class="nv">bundlesequence</span><span class="w"> </span><span class="o">=</span>><span class="w"> </span><span class="o">{</span><span class="w"> </span><span class="s2">"main"</span>,<span class="w"> </span><span class="s2">"test"</span><span class="w"> </span><span class="o">}</span><span class="p">;</span><span class="w"> </span><span class="c1"># 将test加入的承诺束队列</span>
<span class="w"> </span><span class="nv">inputs</span><span class="w"> </span><span class="o">=</span>><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="s2">"cfengine_stdlib.cf"</span>,
<span class="w"> </span><span class="s2">"cftest1.cf"</span>,<span class="w"> </span><span class="c1"># 将cftest1.cf引入进来</span>
<span class="w"> </span><span class="o">}</span><span class="p">;</span>
<span class="w"> </span><span class="nv">version</span><span class="w"> </span><span class="o">=</span>><span class="w"> </span><span class="s2">"Community Promises.cf 1.0.0"</span><span class="p">;</span>
<span class="o">}</span>
.......
</code></pre></div>
<p>做好之后先本地同步,然后启动server:</p>
<pre>/var/cfengine/bin/cf-agent --bootstrap --policy-server 172.16.1.1
<div class="highlight"><pre><span></span><code><span class="err">到如下提示表示成功</span><span class="o">:</span>
<span class="err">```</span><span class="n">bash</span>
<span class="o">-></span><span class="w"> </span><span class="n">Bootstrap</span><span class="w"> </span><span class="n">to</span><span class="w"> </span><span class="mf">172.16.1.1</span><span class="w"> </span><span class="n">completed</span><span class="w"> </span><span class="n">successfully</span>
<span class="n">netstat</span><span class="w"> </span><span class="o">-</span><span class="n">antlp</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">grep</span><span class="w"> </span><span class="n">cf</span>
<span class="n">tcp</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="o">:::</span><span class="mi">5308</span><span class="w"> </span><span class="o">:::*</span><span class="w"> </span><span class="n">LISTEN</span><span class="w"> </span><span class="mi">20173</span><span class="o">/</span><span class="n">cf</span><span class="o">-</span><span class="n">serverd</span>
</code></pre></div>
然后执行策略:
<div class="highlight"><pre><span></span><code>/var/cfengine/bin/cf-agent
R:<span class="w"> </span>--><span class="w"> </span>CFE<span class="w"> </span>is<span class="w"> </span>running<span class="w"> </span>on<span class="w"> </span>cfhub
R:<span class="w"> </span>I<span class="err">'</span>am<span class="w"> </span>cfengine<span class="w"> </span><span class="m">3</span><span class="w"> </span>client
</code></pre></div>
cfengine默认是让本机地址的16位网络连接同步的,如果新加入一个另一网段的设备允许同步,比如允许192.168.1.网段同步,编辑/var/cfengine/masterfiles/promises.cf,找到bundle common def
<div class="highlight"><pre><span></span><code>vi<span class="w"> </span>/var/cfengine/masterfiles/promises.cf
bundle<span class="w"> </span>common<span class="w"> </span>def
<span class="o">{</span>
<span class="w"> </span>vars:<span class="w"> </span><span class="c1"># 定义变量</span>
<span class="w"> </span>...<span class="w"> </span>...
<span class="w"> </span><span class="s2">"acl"</span><span class="w"> </span><span class="nv">slist</span><span class="w"> </span><span class="o">=</span>><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="s2">"</span><span class="k">$(</span>sys.policy_hub<span class="k">)</span><span class="s2">/16"</span>,<span class="w"> </span><span class="s2">"192.168.1."</span>,<span class="w"> </span><span class="c1"># 这这个后面添加</span>
<span class="w"> </span>...<span class="w"> </span>...
<span class="o">}</span>
</code></pre></div>
服务端基本配置完成,客户端(cfengine host),按照本文前面安装的部分进行安装,创建工作目录,复制二进制文件,不用创建配置文件和承诺,然后执行:
<div class="highlight"><pre><span></span><code>/var/cfengine/bin/cf-agent<span class="w"> </span>--bootstrap<span class="w"> </span>--policy-server<span class="w"> </span><span class="m">172</span>.16.1.1
</code></pre></div>
会出现和服务器执行一样的提示,就表示成功,如果提示连接不成功,尝试关闭防火墙.同步后执行本地策略:
<div class="highlight"><pre><span></span><code>/var/cfengine/bin/cf-agent
</code></pre></div>
cfengine官方还是建议在生产环境加一个版本控制器用来创建承诺文件,然后给策略分发点用来分发给策略执行点.更多cfengine的语法请参见手册.Linux shell脚本调试技巧2012-01-17T17:04:00+08:002012-01-17T17:04:00+08:00coldtag:www.linuxzen.com,2012-01-17:/linux-shelljiao-ben-diao-shi-ji-qiao.html<p>有时候shell脚本不会给予明显的调试信息,而且有时不报错,但是脚本没有达到预期的效果这时候脚本调试就可以帮你准确定位错误.</p>
<p>在脚本的最顶部加 …</p><p>有时候shell脚本不会给予明显的调试信息,而且有时不报错,但是脚本没有达到预期的效果这时候脚本调试就可以帮你准确定位错误.</p>
<p>在脚本的最顶部加上</p>
<div class="highlight"><pre><span></span><code><span class="nb">set</span><span class="w"> </span>-x
</code></pre></div>
<p>开启调试
在脚本的最底部加上</p>
<div class="highlight"><pre><span></span><code><span class="nb">set</span><span class="w"> </span>+x
</code></pre></div>
<p>关闭调试
如果在终端界面下,调试信息过多,调试信息是无法重定向到文件的,所以该怎样查看调试呢,当然这对SecureCRT连接的当然不是问题,但是如果是终端界面的话,可以使用命令</p>
<p> </p>
<div class="highlight"><pre><span></span><code>script
</code></pre></div>
<p>然后执行要捕捉内容的命令,完成后通过</p>
<div class="highlight"><pre><span></span><code><span class="nb">exit</span>
</code></pre></div>
<p>退出,当前目录下会生成typescript,通过</p>
<div class="highlight"><pre><span></span><code>more<span class="w"> </span>typescript
</code></pre></div>
<p>查看调试命令.</p>