Multi-Agent-AI-bidding-master.zip

  • benjyhaha
    了解作者
  • Python
    开发工具
  • 962KB
    文件大小
  • zip
    文件格式
  • 0
    收藏次数
  • 1 积分
    下载积分
  • 4
    下载次数
  • 2020-02-19 10:36
    上传日期
是一个多代理系统的交易仿真环境,有不同的交易策略和算法,用户可以自定义,可以用来交易市场仿真。作者Fran?ois-Xavier Aubet, Sylvanus Quek和Sorawit Saengkyongam授权为开源。
Multi-Agent-AI-bidding-master.zip
  • Multi-Agent-AI-bidding-master
  • utils
  • Transformer.py
    5KB
  • utils.py
    674B
  • CEMethod.py
    3.8KB
  • DataHandler.py
    2.6KB
  • algorithms
  • BidStream.py
    7.5KB
  • ConstantBid.py
    2.5KB
  • DictReader.py
    1.2KB
  • RandomBid.py
    2.9KB
  • PRUD.py
    3KB
  • LinearBid.py
    5.4KB
  • PolicyGradient.py
    5.8KB
  • IAlgorithm.py
    1.3KB
  • CEEnsemble.py
    2.3KB
  • CDFBid.py
    8.9KB
  • notebooks
  • ensemble.ipynb
    22.3KB
  • linear_submission-xgboost.ipynb
    20.5KB
  • CDF-logistic.ipynb
    9.5KB
  • eval_CTRlogistic.ipynb
    13.5KB
  • CDF-XGB.ipynb
    56KB
  • eval_CTRXGboost.ipynb
    16.9KB
  • linear_submission.ipynb
    15.7KB
  • models
  • IModel.py
    820B
  • CTRXGboost.py
    4KB
  • MPXGBoost.py
    3.7KB
  • CTRLogistic.py
    5.5KB
  • plot_docs
  • Figure_2.pdf
    24.2KB
  • final_budgets.pdf
    76.7KB
  • cropAll.sh
    128B
  • Figure_3.pdf
    23.7KB
  • Figure_1.png
    52.2KB
  • final_compete.pdf
    23.6KB
  • factors_time.pdf
    124.4KB
  • evaluators
  • Metrics.py
    3.7KB
  • Evaluator.py
    3.2KB
  • MultiEvaluatorOff.py
    3.9KB
  • MultiEvalualOnline.py
    5.3KB
  • MAAI_report.pdf
    676.3KB
  • .gitignore
    2.3KB
内容介绍
<html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta charset="utf-8"> <meta name="generator" content="pdf2htmlEX"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <link rel="stylesheet" href="https://static.pudn.com/base/css/base.min.css"> <link rel="stylesheet" href="https://static.pudn.com/base/css/fancy.min.css"> <link rel="stylesheet" href="https://static.pudn.com/prod/directory_preview_static/6251315174bc5c010595f88b/raw.css"> <script src="https://static.pudn.com/base/js/compatibility.min.js"></script> <script src="https://static.pudn.com/base/js/pdf2htmlEX.min.js"></script> <script> try{ pdf2htmlEX.defaultViewer = new pdf2htmlEX.Viewer({}); }catch(e){} </script> <title></title> </head> <body> <div id="sidebar" style="display: none"> <div id="outline"> </div> </div> <div id="pf1" class="pf w0 h0" data-page-no="1"><div class="pc pc1 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/6251315174bc5c010595f88b/bg1.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">Multi-Agent<span class="_"> </span>Ar<span class="_ _0"></span>ti&#64257;cial<span class="_"> </span>Intelligence<span class="_"> </span>Project<span class="_"> </span>(Gr<span class="_ _1"></span>oup<span class="_"> </span>3)</div><div class="t m0 x2 h3 y2 ff2 fs1 fc0 sc0 ls0 ws0">F<span class="_ _1"></span>ran&#231;ois-Xa<span class="_ _1"></span>vier<span class="_ _2"> </span>A<span class="_ _1"></span>ubet</div><div class="t m0 x3 h4 y3 ff3 fs2 fc0 sc0 ls0 ws0">&#8727;</div><div class="t m0 x4 h3 y2 ff2 fs1 fc0 sc0 ls0 ws0">Sylv<span class="_ _1"></span>anus<span class="_ _2"> </span>Quek</div><div class="t m0 x5 h4 y3 ff3 fs2 fc0 sc0 ls0 ws0">&#8727;</div><div class="t m0 x6 h3 y2 ff2 fs1 fc0 sc0 ls0 ws0">Sora<span class="_ _1"></span>wit<span class="_ _2"> </span>Saengkyongam</div><div class="t m0 x7 h4 y3 ff3 fs2 fc0 sc0 ls0 ws0">&#8727;</div><div class="t m0 x8 h5 y4 ff4 fs1 fc0 sc0 ls0 ws0">1.<span class="_ _3"> </span>INTR<span class="_ _1"></span>ODUCTION</div><div class="t m0 x9 h6 y5 ff5 fs3 fc0 sc0 ls0 ws0">Since<span class="_"> </span>2009,<span class="_ _4"> </span>computational<span class="_"> </span>advertisemen<span class="_ _1"></span>t<span class="_"> </span>has<span class="_ _4"> </span>increasingly</div><div class="t m0 x8 h6 y6 ff5 fs3 fc0 sc0 ls0 ws0">been<span class="_"> </span>conducted<span class="_"> </span>through<span class="_"> </span>Real-Time<span class="_"> </span>Bidding<span class="_"> </span>(R<span class="_ _5"></span>TB).<span class="_"> </span>[2]<span class="_"> </span>This</div><div class="t m0 x8 h6 y7 ff5 fs3 fc0 sc0 ls0 ws0">system<span class="_ _6"> </span>allo<span class="_ _7"></span>ws<span class="_ _6"> </span>publishers<span class="_ _6"> </span>to<span class="_ _6"> </span>sell<span class="_ _6"> </span>individual<span class="_ _6"> </span>ad<span class="_ _6"> </span>impressions</div><div class="t m0 x8 h6 y8 ff5 fs3 fc0 sc0 ls0 ws0">to<span class="_ _2"> </span>adv<span class="_ _7"></span>ertisers<span class="_ _2"> </span>using<span class="_ _2"> </span>real-time<span class="_ _2"> </span>auctions.<span class="_ _8"> </span>The<span class="_ _2"> </span>adv<span class="_ _7"></span>ertisers<span class="_ _2"> </span>all</div><div class="t m0 x8 h6 y9 ff5 fs3 fc0 sc0 ls0 ws0">receiv<span class="_ _7"></span>e<span class="_ _9"> </span>a<span class="_ _4"> </span>description<span class="_ _9"> </span>of<span class="_ _9"> </span>the<span class="_ _9"> </span>ad<span class="_ _9"> </span>impression<span class="_ _9"> </span>av<span class="_ _5"></span>ailable<span class="_ _9"> </span>for<span class="_ _9"> </span>auc-</div><div class="t m0 x8 h6 ya ff5 fs3 fc0 sc0 ls0 ws0">tion<span class="_ _6"> </span>whic<span class="_ _1"></span>h<span class="_ _6"> </span>allo<span class="_ _1"></span>ws<span class="_ _6"> </span>them<span class="_ _6"> </span>to<span class="_ _a"> </span>ev<span class="_ _1"></span>aluate<span class="_ _a"> </span>and<span class="_ _6"> </span>decide<span class="_ _a"> </span>how<span class="_ _a"> </span>muc<span class="_ _1"></span>h</div><div class="t m0 x8 h6 yb ff5 fs3 fc0 sc0 ls0 ws0">to<span class="_ _9"> </span>bid<span class="_ _2"> </span>on<span class="_ _2"> </span>it.<span class="_ _b"> </span>The<span class="_ _9"> </span>ad<span class="_ _2"> </span>exchange<span class="_ _9"> </span>gathers<span class="_ _2"> </span>the<span class="_ _9"> </span>bids<span class="_ _2"> </span>and<span class="_ _9"> </span>selects</div><div class="t m0 x8 h6 yc ff5 fs3 fc0 sc0 ls0 ws0">as<span class="_ _9"> </span>winner<span class="_ _2"> </span>the<span class="_ _2"> </span>one<span class="_ _9"> </span>with<span class="_ _2"> </span>the<span class="_ _9"> </span>highest<span class="_ _2"> </span>bid<span class="_ _2"> </span>price.<span class="_ _b"> </span>The<span class="_ _9"> </span>winning</div><div class="t m0 x8 h6 yd ff5 fs3 fc0 sc0 ls0 ws0">adv<span class="_ _7"></span>ertiser<span class="_"> </span>is<span class="_"> </span>then<span class="_"> </span>p<span class="_ _0"></span>ermitted<span class="_"> </span>to<span class="_"> </span>sho<span class="_ _1"></span>w<span class="_"> </span>its<span class="_"> </span>ad.<span class="_ _a"> </span>In<span class="_ _4"> </span>this<span class="_"> </span>w<span class="_ _7"></span>ork,<span class="_"> </span>we</div><div class="t m0 x8 h6 ye ff5 fs3 fc0 sc0 ls0 ws0">only<span class="_ _4"> </span>consider<span class="_ _4"> </span>second-price<span class="_ _4"> </span>auctions<span class="_ _4"> </span>where<span class="_ _4"> </span>the<span class="_ _9"> </span>winning<span class="_ _4"> </span>pays</div><div class="t m0 x8 h6 yf ff5 fs3 fc0 sc0 ls0 ws0">the<span class="_ _2"> </span>price<span class="_ _a"> </span>of<span class="_ _2"> </span>the<span class="_ _a"> </span>second<span class="_ _2"> </span>highest<span class="_ _a"> </span>bid.<span class="_ _c"> </span>The<span class="_ _2"> </span>main<span class="_ _a"> </span>goal<span class="_ _2"> </span>for<span class="_ _a"> </span>an</div><div class="t m0 x8 h6 y10 ff5 fs3 fc0 sc0 ls0 ws0">adv<span class="_ _7"></span>ertiser<span class="_ _4"> </span>is<span class="_ _4"> </span>to<span class="_ _9"> </span>maximize<span class="_ _4"> </span>the<span class="_ _9"> </span>n<span class="_ _7"></span>umber<span class="_ _4"> </span>of<span class="_ _4"> </span>clicks<span class="_ _4"> </span>on<span class="_ _4"> </span>the<span class="_ _9"> </span>ads<span class="_ _4"> </span>it</div><div class="t m0 x8 h6 y11 ff5 fs3 fc0 sc0 ls0 ws0">displa<span class="_ _7"></span>ys.<span class="_ _d"> </span>This<span class="_ _2"> </span>demands<span class="_ _2"> </span>an<span class="_ _a"> </span>accurate<span class="_ _2"> </span>assessmen<span class="_ _1"></span>t<span class="_ _a"> </span>of<span class="_ _2"> </span>the<span class="_ _2"> </span>im-</div><div class="t m0 x8 h6 y12 ff5 fs3 fc0 sc0 ls0 ws0">pressions&#8217;<span class="_ _2"> </span>descriptions<span class="_ _2"> </span>and<span class="_ _2"> </span>estimation<span class="_ _2"> </span>of<span class="_ _a"> </span>ho<span class="_ _1"></span>w<span class="_ _a"> </span>lik<span class="_ _1"></span>ely<span class="_ _a"> </span>the<span class="_ _2"> </span>ad</div><div class="t m0 x8 h6 y13 ff5 fs3 fc0 sc0 ls0 ws0">w<span class="_ _7"></span>ould<span class="_ _4"> </span>b<span class="_ _0"></span>e<span class="_ _4"> </span>click<span class="_ _1"></span>ed<span class="_ _9"> </span>on,<span class="_ _4"> </span>i.e.<span class="_ _6"> </span>Click<span class="_ _4"> </span>Through<span class="_ _4"> </span>Rate<span class="_ _9"> </span>(CTR)<span class="_ _4"> </span>[12].</div><div class="t m0 x9 h6 y14 ff5 fs3 fc0 sc0 ls0 ws0">In<span class="_ _6"> </span>this<span class="_ _6"> </span>work,<span class="_ _e"> </span>w<span class="_ _7"></span>e<span class="_ _6"> </span>use<span class="_ _6"> </span>ad<span class="_ _e"> </span>impressions<span class="_ _6"> </span>collected<span class="_ _6"> </span>from<span class="_ _6"> </span>the</div><div class="t m0 x8 h6 y15 ff5 fs3 fc0 sc0 ls0 ws0">iPinY<span class="_ _5"></span>ou<span class="_ _6"> </span>logs<span class="_ _a"> </span>[20]<span class="_ _6"> </span>to<span class="_ _a"> </span>design<span class="_ _6"> </span>and<span class="_ _a"> </span>ev<span class="_ _1"></span>aluate<span class="_ _6"> </span>di&#64256;erent<span class="_ _a"> </span>bidding</div><div class="t m0 x8 h6 y16 ff5 fs3 fc0 sc0 ls0 ws0">strategies.<span class="_ _b"> </span>W<span class="_ _5"></span>e<span class="_ _2"> </span>ev<span class="_ _1"></span>aluate<span class="_ _9"> </span>the<span class="_ _2"> </span>strategies<span class="_ _2"> </span>on<span class="_ _9"> </span>the<span class="_ _2"> </span>v<span class="_ _1"></span>alidation<span class="_ _9"> </span>set</div><div class="t m0 x8 h6 y17 ff5 fs3 fc0 sc0 ls0 ws0">using<span class="_ _f"> </span>a<span class="_ _f"> </span>&#64257;xed<span class="_ _f"> </span>budget<span class="_ _f"> </span>of<span class="_ _f"> </span>6,250<span class="_ _f"> </span>CNY<span class="_ _f"> </span>fen<span class="_ _f"> </span>under<span class="_ _f"> </span>tw<span class="_ _1"></span>o<span class="_ _f"> </span>distinct<span class="_ _f"> </span>set-</div><div class="t m0 x8 h6 y18 ff5 fs3 fc0 sc0 ls0 ws0">tings.<span class="_ _e"> </span>First,<span class="_ _9"> </span>the<span class="_ _2"> </span>algorithms<span class="_ _9"> </span>compete<span class="_ _2"> </span>against<span class="_ _9"> </span>the<span class="_ _9"> </span>pay<span class="_ _4"> </span>prices</div><div class="t m0 x8 h6 y19 ff5 fs3 fc0 sc0 ls0 ws0">of<span class="_ _a"> </span>the<span class="_ _2"> </span>impressions<span class="_ _a"> </span>collected<span class="_ _a"> </span>in<span class="_ _2"> </span>the<span class="_ _a"> </span>data<span class="_ _a"> </span>set<span class="_ _a"> </span>giving<span class="_ _2"> </span>us<span class="_ _a"> </span>win-</div><div class="t m0 x8 h6 y1a ff5 fs3 fc0 sc0 ls0 ws0">ning<span class="_ _6"> </span>criteria<span class="_ _6"> </span>#1:<span class="_ _c"> </span>auction<span class="_ _6"> </span>is<span class="_ _6"> </span>won<span class="_ _a"> </span>if<span class="_ _6"> </span><span class="ff6">bid<span class="_ _e"> </span><span class="ff7">&#8805;<span class="_ _e"> </span></span>pay<span class="_ _0"></span>price</span>.<span class="_ _10"> </span>Sec-</div><div class="t m0 x8 h6 y1b ff5 fs3 fc0 sc0 ls0 ws0">ond,<span class="_"> </span>the<span class="_"> </span>algorithms<span class="_ _4"> </span>compete<span class="_ _4"> </span>against<span class="_"> </span>one<span class="_"> </span>another<span class="_"> </span>using<span class="_"> </span>win-</div><div class="t m0 x8 h6 y1c ff5 fs3 fc0 sc0 ls0 ws0">ning<span class="_ _6"> </span>criteria<span class="_ _e"> </span>#2:<span class="_ _11"> </span>auction<span class="_ _6"> </span>is<span class="_ _e"> </span>won<span class="_ _6"> </span>if<span class="_ _e"> </span><span class="ff6">bid<span class="_ _b"> </span><span class="ff7">&#8805;<span class="_ _b"> </span></span></span>max(<span class="ff6">paypr<span class="_ _0"></span>ice,</span></div><div class="t m0 x8 h6 y1d ff6 fs3 fc0 sc0 ls0 ws0">other<span class="_ _12"></span>S<span class="_ _0"></span>ubmittedB<span class="_ _12"></span>ids<span class="ff5">).</span></div><div class="t m0 x9 h6 y1e ff5 fs3 fc0 sc0 ls0 ws0">W<span class="_ _5"></span>e<span class="_ _4"> </span>&#64257;rst<span class="_ _4"> </span>review<span class="_ _4"> </span>related<span class="_ _4"> </span>work<span class="_"> </span>and<span class="_ _4"> </span>perform<span class="_ _4"> </span>a<span class="_ _4"> </span>detailed<span class="_ _4"> </span>anal-</div><div class="t m0 x8 h6 y1f ff5 fs3 fc0 sc0 ls0 ws0">ysis<span class="_"> </span>of<span class="_"> </span>the<span class="_"> </span>data<span class="_"> </span>set.<span class="_ _6"> </span>F<span class="_ _5"></span>ollowing<span class="_"> </span>that,<span class="_"> </span>basic<span class="_"> </span>baseline<span class="_"> </span>strategies</div><div class="t m0 x8 h6 y20 ff5 fs3 fc0 sc0 ls0 ws0">are<span class="_ _a"> </span>created<span class="_ _6"> </span>and<span class="_ _a"> </span>we<span class="_ _a"> </span>inv<span class="_ _1"></span>estigate<span class="_ _6"> </span>the<span class="_ _a"> </span>multi-agen<span class="_ _1"></span>t<span class="_ _6"> </span>setting<span class="_ _a"> </span>for</div><div class="t m0 x8 h6 y21 ff5 fs3 fc0 sc0 ls0 ws0">one<span class="_ _9"> </span>of<span class="_ _9"> </span>them.<span class="_ _6"> </span>In<span class="_ _9"> </span>the<span class="_ _9"> </span>later<span class="_ _9"> </span>sections,<span class="_ _9"> </span>we<span class="_ _4"> </span>develop<span class="_ _4"> </span>and<span class="_ _9"> </span>ev<span class="_ _1"></span>aluate</div><div class="t m0 x8 h6 y22 ff5 fs3 fc0 sc0 ls0 ws0">linear<span class="_ _9"> </span>and<span class="_ _2"> </span>non-linear<span class="_ _9"> </span>bidding<span class="_ _2"> </span>strategies.<span class="_ _e"> </span>Finally<span class="_ _1"></span>,<span class="_ _9"> </span>we<span class="_ _9"> </span>design</div><div class="t m0 x8 h6 y23 ff5 fs3 fc0 sc0 ls0 ws0">a<span class="_ _2"> </span>multi-a<span class="_ _7"></span>gent<span class="_ _9"> </span>and<span class="_ _2"> </span>compare<span class="_ _2"> </span>all<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _a"> </span>strategies<span class="_ _2"> </span>in<span class="_ _2"> </span>a<span class="_ _2"> </span>m<span class="_ _1"></span>ulti-</div><div class="t m0 x8 h6 y24 ff5 fs3 fc0 sc0 ls0 ws0">agen<span class="_ _7"></span>t<span class="_ _2"> </span>environmen<span class="_ _1"></span>t.<span class="_ _d"> </span>The<span class="_ _2"> </span>implementation<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _2"> </span>algorithms</div><div class="t m0 x8 h6 y25 ff5 fs3 fc0 sc0 ls0 ws0">created<span class="_ _4"> </span>in<span class="_ _9"> </span>this<span class="_ _4"> </span>pap<span class="_ _12"></span>er<span class="_ _9"> </span>is<span class="_ _9"> </span>made<span class="_ _4"> </span>publicly<span class="_ _9"> </span>av<span class="_ _1"></span>ailable.</div><div class="t m0 xa h7 y26 ff8 fs2 fc0 sc0 ls0 ws0">1</div><div class="t m0 x8 h5 y27 ff4 fs1 fc0 sc0 ls0 ws0">2.<span class="_ _3"> </span>LITERA<span class="_ _5"></span>TURE<span class="_ _4"> </span>REVIEW</div><div class="t m0 x9 h6 y28 ff5 fs3 fc0 sc0 ls0 ws0">In<span class="_ _6"> </span>this<span class="_ _a"> </span>section,<span class="_ _e"> </span>we<span class="_ _a"> </span>review<span class="_ _6"> </span>v<span class="_ _1"></span>arious<span class="_ _6"> </span>previous<span class="_ _a"> </span>work<span class="_ _6"> </span>on<span class="_ _a"> </span>as-</div><div class="t m0 x8 h6 y29 ff5 fs3 fc0 sc0 ls0 ws0">pects<span class="_ _a"> </span>of<span class="_ _2"> </span>R<span class="_ _1"></span>TB.<span class="_ _2"> </span>The<span class="_ _a"> </span>early<span class="_ _2"> </span>works<span class="_ _2"> </span>in<span class="_ _2"> </span>R<span class="_ _1"></span>TB<span class="_ _2"> </span>fo<span class="_ _12"></span>cused<span class="_ _a"> </span>mainly<span class="_ _2"> </span>on</div><div class="t m0 x8 h6 y2a ff5 fs3 fc0 sc0 ls0 ws0">utilit<span class="_ _7"></span>y<span class="_ _4"> </span>estimation<span class="_"> </span>and<span class="_ _4"> </span>utility-based<span class="_"> </span>bidding<span class="_ _4"> </span>strategies.<span class="_ _a"> </span>Util-</div><div class="t m0 x8 h6 y2b ff5 fs3 fc0 sc0 ls0 ws0">it<span class="_ _7"></span>y<span class="_"> </span>estimation<span class="_"> </span>often<span class="_ _f"> </span>b<span class="_ _12"></span>oils<span class="_"> </span>down<span class="_ _f"> </span>to<span class="_"> </span>a<span class="_"> </span>user<span class="_"> </span>response<span class="_"> </span>prediction</div><div class="t m0 x8 h6 y2c ff5 fs3 fc0 sc0 ls0 ws0">suc<span class="_ _7"></span>h<span class="_ _9"> </span>as<span class="_ _9"> </span>CTR<span class="_ _9"> </span>and<span class="_ _9"> </span>con<span class="_ _7"></span>version<span class="_ _4"> </span>rate<span class="_ _9"> </span>(CVR)<span class="_ _9"> </span>predictions.<span class="_ _6"> </span>CTR</div><div class="t m0 x8 h6 y2d ff5 fs3 fc0 sc0 ls0 ws0">and<span class="_ _9"> </span>CVR<span class="_ _2"> </span>predictions<span class="_ _9"> </span>are<span class="_ _2"> </span>typically<span class="_ _9"> </span>framed<span class="_ _9"> </span>as<span class="_ _2"> </span>binary<span class="_ _2"> </span>classi-</div><div class="t m0 x8 h6 y2e ff5 fs3 fc0 sc0 ls0 ws0">&#64257;cation<span class="_ _2"> </span>problems.<span class="_ _8"> </span>Several<span class="_ _9"> </span>machine<span class="_ _2"> </span>learning<span class="_ _2"> </span>algorithms<span class="_ _2"> </span>for</div><div class="t m0 x8 h6 y2f ff5 fs3 fc0 sc0 ls0 ws0">classi&#64257;cation<span class="_ _4"> </span>suc<span class="_ _7"></span>h<span class="_ _4"> </span>as<span class="_ _4"> </span>logistic<span class="_ _4"> </span>regression<span class="_ _4"> </span>[7],<span class="_ _4"> </span>Gradient<span class="_ _4"> </span>Bo<span class="_ _12"></span>ost-</div><div class="t m0 x8 h6 y30 ff5 fs3 fc0 sc0 ls0 ws0">ing<span class="_ _a"> </span>T<span class="_ _5"></span>rees<span class="_ _a"> </span>[14]<span class="_ _a"> </span>and<span class="_ _a"> </span>F<span class="_ _5"></span>actorization<span class="_ _a"> </span>Machine<span class="_ _2"> </span>[6]<span class="_ _a"> </span>hav<span class="_ _1"></span>e<span class="_ _a"> </span>success-</div><div class="t m0 x8 h6 y31 ff5 fs3 fc0 sc0 ls0 ws0">fully<span class="_ _4"> </span>been<span class="_ _4"> </span>applied<span class="_ _4"> </span>in<span class="_ _4"> </span>utility<span class="_"> </span>estimation.<span class="_ _6"> </span>With<span class="_ _4"> </span>the<span class="_ _4"> </span>estimated</div><div class="t m0 x8 h6 y32 ff5 fs3 fc0 sc0 ls0 ws0">utilit<span class="_ _7"></span>y<span class="_ _5"></span>,<span class="_ _4"> </span>utility-based<span class="_"> </span>bidding<span class="_"> </span>strategies<span class="_ _4"> </span>then<span class="_ _4"> </span>exploit<span class="_"> </span>the<span class="_ _4"> </span>util-</div><div class="t m0 x8 h6 y33 ff5 fs3 fc0 sc0 ls0 ws0">it<span class="_ _7"></span>y<span class="_ _6"> </span>prediction<span class="_ _6"> </span>to<span class="_ _6"> </span>p<span class="_ _12"></span>erform<span class="_ _6"> </span>a<span class="_ _6"> </span>bid.<span class="_ _10"> </span>Auction<span class="_ _6"> </span>theory<span class="_ _6"> </span>suggests</div><div class="t m0 x8 h6 y34 ff5 fs3 fc0 sc0 ls0 ws0">that<span class="_ _a"> </span>truthful<span class="_ _a"> </span>bidding,<span class="_ _6"> </span>where<span class="_ _a"> </span>w<span class="_ _7"></span>e<span class="_ _a"> </span>bid<span class="_ _a"> </span>according<span class="_ _a"> </span>to<span class="_ _a"> </span>the<span class="_ _a"> </span>true</div><div class="t m0 x8 h8 y35 ff9 fs2 fc0 sc0 ls0 ws0">&#8727;</div><div class="t m0 xb h6 y36 ff5 fs3 fc0 sc0 ls0 ws0">All<span class="_ _4"> </span>authors<span class="_ _9"> </span>contribute<span class="_ _7"></span>d<span class="_ _4"> </span>equally<span class="_ _9"> </span>to<span class="_ _9"> </span>this<span class="_ _4"> </span>work.</div><div class="t m0 x8 h7 y37 ff8 fs2 fc0 sc0 ls0 ws0">1</div><div class="t m0 xc h6 y38 ff5 fs3 fc0 sc0 ls0 ws0">h<span class="_ _7"></span>ttps://gith<span class="_ _1"></span>ub.com/F<span class="_ _1"></span>rancois-<span class="_ _12"></span>Aub<span class="_ _12"></span>et/</div><div class="t m0 x8 h6 y39 ff5 fs3 fc0 sc0 ls0 ws0">Multi-<span class="_ _12"></span>Agent-<span class="_ _12"></span>AI-<span class="_ _0"></span>bidding</div><div class="t m0 xd h6 y4 ff5 fs3 fc0 sc0 ls0 ws0">utilit<span class="_ _7"></span>y<span class="_ _e"> </span>v<span class="_ _1"></span>alue<span class="_ _e"> </span>of<span class="_ _e"> </span>an<span class="_ _e"> </span>impression,<span class="_ _b"> </span>is<span class="_ _e"> </span>the<span class="_ _e"> </span>optimal<span class="_ _e"> </span>strategy<span class="_ _e"> </span>in</div><div class="t m0 xd h6 y3a ff5 fs3 fc0 sc0 ls0 ws0">second<span class="_ _2"> </span>price<span class="_ _9"> </span>auction<span class="_ _2"> </span>[7].<span class="_ _8"> </span>How<span class="_ _1"></span>ever,<span class="_ _2"> </span>truthful<span class="_ _9"> </span>bidding<span class="_ _2"> </span>can<span class="_ _2"> </span>b<span class="_ _12"></span>e</div><div class="t m0 xd h6 y3b ff5 fs3 fc0 sc0 ls0 ws0">sub-optimal<span class="_ _2"> </span>if<span class="_ _a"> </span>we<span class="_ _2"> </span>consider<span class="_ _a"> </span>budget<span class="_ _a"> </span>and<span class="_ _2"> </span>volume<span class="_ _2"> </span>(num<span class="_ _1"></span>b<span class="_ _12"></span>er<span class="_ _a"> </span>of</div><div class="t m0 xd h6 y3c ff5 fs3 fc0 sc0 ls0 ws0">impressions)<span class="_ _2"> </span>constraints<span class="_ _2"> </span>[19].<span class="_ _13"> </span>The<span class="_ _2"> </span>non-truthful<span class="_ _a"> </span>linear<span class="_ _a"> </span>bid-</div><div class="t m0 xd h6 y3d ff5 fs3 fc0 sc0 ls0 ws0">ding<span class="_"> </span>strategy<span class="_ _4"> </span>was<span class="_"> </span>then<span class="_ _4"> </span>proposed<span class="_ _4"> </span>by<span class="_"> </span>[19]<span class="_"> </span>where<span class="_ _4"> </span>the<span class="_ _4"> </span>bid<span class="_ _4"> </span>price</div><div class="t m0 xd h6 y3e ff5 fs3 fc0 sc0 ls0 ws0">is<span class="_ _4"> </span>calculated<span class="_ _4"> </span>by<span class="_ _4"> </span>CTR/CVR<span class="_ _4"> </span>estimation<span class="_ _4"> </span>multiplied<span class="_ _4"> </span>b<span class="_ _7"></span>y<span class="_ _4"> </span>a<span class="_ _9"> </span>con-</div><div class="t m0 xd h6 y3f ff5 fs3 fc0 sc0 ls0 ws0">stan<span class="_ _7"></span>t<span class="_"> </span>parameter<span class="_ _f"> </span>which<span class="_ _f"> </span>is<span class="_"> </span>tuned<span class="_ _f"> </span>according<span class="_"> </span>to<span class="_ _f"> </span>the<span class="_"> </span>budget<span class="_"> </span>c<span class="_ _7"></span>on-</div><div class="t m0 xd h6 y40 ff5 fs3 fc0 sc0 ls0 ws0">strain<span class="_ _7"></span>t.<span class="_ _6"> </span>Later<span class="_ _9"> </span>on,<span class="_ _9"> </span>[19]<span class="_ _9"> </span>suggested<span class="_ _4"> </span>a<span class="_ _9"> </span>non-linear<span class="_ _9"> </span>bidding<span class="_ _9"> </span>strat-</div><div class="t m0 xd h6 y41 ff5 fs3 fc0 sc0 ls0 ws0">egy<span class="_ _5"></span>.<span class="_ _11"> </span>The<span class="_ _6"> </span>authors<span class="_ _a"> </span>derived<span class="_ _a"> </span>a<span class="_ _a"> </span>closed-form<span class="_ _6"> </span>solution<span class="_ _a"> </span>in<span class="_ _6"> </span>whic<span class="_ _1"></span>h</div><div class="t m0 xd h6 y42 ff5 fs3 fc0 sc0 ls0 ws0">the<span class="_ _2"> </span>bidding<span class="_ _9"> </span>function<span class="_ _2"> </span>is<span class="_ _2"> </span>a<span class="_ _2"> </span>conca<span class="_ _1"></span>ve<span class="_ _2"> </span>function<span class="_ _9"> </span>instead<span class="_ _2"> </span>of<span class="_ _2"> </span>a<span class="_ _2"> </span>lin-</div><div class="t m0 xd h6 y43 ff5 fs3 fc0 sc0 ls0 ws0">ear<span class="_ _4"> </span>function.<span class="_ _6"> </span>A<span class="_ _9"> </span>lift-based<span class="_ _9"> </span>bidding<span class="_ _9"> </span>strategy<span class="_ _9"> </span>was<span class="_ _4"> </span>proposed<span class="_ _9"> </span>in</div><div class="t m0 xd h6 y44 ff5 fs3 fc0 sc0 ls0 ws0">[16]<span class="_ _2"> </span>where<span class="_ _2"> </span>the<span class="_ _2"> </span>bid<span class="_ _2"> </span>price<span class="_ _2"> </span>is<span class="_ _2"> </span>decided<span class="_ _2"> </span>based<span class="_ _2"> </span>on<span class="_ _2"> </span>the<span class="_ _2"> </span>utilit<span class="_ _1"></span>y<span class="_ _a"> </span>lift</div><div class="t m0 xd h6 y45 ff5 fs3 fc0 sc0 ls0 ws0">rather<span class="_ _4"> </span>than<span class="_ _9"> </span>absolute<span class="_ _4"> </span>utility<span class="_ _4"> </span>v<span class="_ _1"></span>alue.</div><div class="t m0 xe h6 y46 ff5 fs3 fc0 sc0 ls0 ws0">Another<span class="_ _4"> </span>line<span class="_ _4"> </span>of<span class="_ _4"> </span>wo<span class="_ _7"></span>rks<span class="_ _4"> </span>inv<span class="_ _1"></span>olve<span class="_"> </span>strategies<span class="_ _4"> </span>that<span class="_ _4"> </span>rely<span class="_ _4"> </span>on<span class="_ _4"> </span>b<span class="_ _12"></span>oth</div><div class="t m0 xd h6 y47 ff5 fs3 fc0 sc0 ls0 ws0">utilit<span class="_ _7"></span>y<span class="_ _4"> </span>and<span class="_ _4"> </span>cost<span class="_"> </span>estimation.<span class="_ _6"> </span>Cost<span class="_ _4"> </span>estimation<span class="_"> </span>aims<span class="_ _4"> </span>to<span class="_ _4"> </span>predict</div><div class="t m0 xd h6 y48 ff5 fs3 fc0 sc0 ls0 ws0">the<span class="_ _f"> </span>mark<span class="_ _1"></span>et<span class="_"> </span>p<span class="_ _7"></span>rice<span class="_ _f"> </span>distribution<span class="_ _f"> </span>giv<span class="_ _1"></span>en<span class="_ _f"> </span>features<span class="_ _f"> </span>of<span class="_ _f"> </span>an<span class="_ _f"> </span>impression.</div><div class="t m0 xd h6 y49 ff5 fs3 fc0 sc0 ls0 ws0">The<span class="_ _4"> </span>challenge<span class="_ _4"> </span>is<span class="_ _9"> </span>that<span class="_ _9"> </span>advertisers<span class="_ _4"> </span>only<span class="_ _9"> </span>gain<span class="_ _9"> </span>market<span class="_ _4"> </span>price<span class="_ _4"> </span>in-</div><div class="t m0 xd h6 y4a ff5 fs3 fc0 sc0 ls0 ws0">formation<span class="_ _a"> </span>from<span class="_ _a"> </span>impressions<span class="_ _a"> </span>won.<span class="_ _14"> </span>Censored<span class="_ _a"> </span>regression<span class="_ _a"> </span>has</div><div class="t m0 xd h6 y4b ff5 fs3 fc0 sc0 ls0 ws0">been<span class="_ _a"> </span>adopted<span class="_ _a"> </span>to<span class="_ _2"> </span>address<span class="_ _a"> </span>this<span class="_ _a"> </span>censoring<span class="_ _2"> </span>problem<span class="_ _a"> </span>[15].<span class="_ _c"> </span>The</div><div class="t m0 xd h6 y4c ff5 fs3 fc0 sc0 ls0 ws0">authors<span class="_ _f"> </span>in<span class="_"> </span>[8]<span class="_ _f"> </span>com<span class="_ _7"></span>bined<span class="_ _f"> </span>cost<span class="_"> </span>and<span class="_ _f"> </span>utilit<span class="_ _7"></span>y<span class="_"> </span>estima<span class="_ _7"></span>tions<span class="_ _f"> </span>to<span class="_"> </span>de&#64257;ne</div><div class="t m0 xd h6 y4d ff5 fs3 fc0 sc0 ls0 ws0">an<span class="_ _a"> </span>e&#64259;ciency<span class="_ _6"> </span>(<span class="ffa">r<span class="_ _1"></span>eturn<span class="_ _6"> </span>on<span class="_ _a"> </span>investment<span class="_ _15"></span><span class="ff5">)<span class="_ _a"> </span>of<span class="_ _6"> </span>an<span class="_ _a"> </span>impression<span class="_ _a"> </span>and</span></span></div><div class="t m0 xd h6 y4e ff5 fs3 fc0 sc0 ls0 ws0">used<span class="_ _4"> </span>it<span class="_ _9"> </span>to<span class="_ _9"> </span>come<span class="_ _9"> </span>up<span class="_ _9"> </span>with<span class="_ _4"> </span>the<span class="_ _9"> </span>&#64257;nal<span class="_ _9"> </span>bid<span class="_ _9"> </span>price.<span class="_ _6"> </span>[13]<span class="_ _4"> </span>prop<span class="_ _12"></span>osed<span class="_ _9"> </span>a</div><div class="t m0 xd h6 y4f ff5 fs3 fc0 sc0 ls0 ws0">join<span class="_ _7"></span>t<span class="_"> </span>optimization<span class="_ _4"> </span>approac<span class="_ _7"></span>h<span class="_"> </span>which<span class="_"> </span>jointly<span class="_"> </span>learns<span class="_"> </span>utility<span class="_"> </span>esti-</div><div class="t m0 xd h6 y50 ff5 fs3 fc0 sc0 ls0 ws0">mation,<span class="_ _4"> </span>cost<span class="_ _9"> </span>estimation<span class="_ _4"> </span>and<span class="_ _9"> </span>bidding<span class="_ _4"> </span>function<span class="_ _9"> </span>as<span class="_ _9"> </span>a<span class="_ _4"> </span>whole.</div><div class="t m0 xe h6 y51 ff5 fs3 fc0 sc0 ls0 ws0">All<span class="_"> </span>of<span class="_ _4"> </span>the<span class="_ _4"> </span>aforemen<span class="_ _7"></span>tioned<span class="_"> </span>works<span class="_"> </span>assume<span class="_ _4"> </span>that<span class="_"> </span>the<span class="_ _4"> </span>environ-</div><div class="t m0 xd h6 y52 ff5 fs3 fc0 sc0 ls0 ws0">men<span class="_ _7"></span>t<span class="_ _9"> </span>is<span class="_ _9"> </span>stationary<span class="_ _1"></span>.<span class="_ _6"> </span>In<span class="_ _9"> </span>recent<span class="_ _9"> </span>y<span class="_ _7"></span>ears,<span class="_ _9"> </span>the<span class="_ _9"> </span>works<span class="_ _4"> </span>in<span class="_ _2"> </span>R<span class="_ _5"></span>TB<span class="_ _9"> </span>hav<span class="_ _1"></span>e</div><div class="t m0 xd h6 y53 ff5 fs3 fc0 sc0 ls0 ws0">piv<span class="_ _7"></span>oted<span class="_ _f"> </span>from<span class="_ _f"> </span>stat<span class="_ _7"></span>ionary<span class="_ _f"> </span>to<span class="_ _f"> </span>non-<span class="_ _7"></span>stationary<span class="_ _f"> </span>en<span class="_ _1"></span>vironments<span class="_ _16"> </span>where</div><div class="t m0 xd h6 y54 ff5 fs3 fc0 sc0 ls0 ws0">the<span class="_"> </span>c<span class="_ _1"></span>hanges<span class="_"> </span>in<span class="_ _16"> </span>the<span class="_"> </span>strategy<span class="_ _f"> </span>of<span class="_"> </span>one<span class="_ _f"> </span>adverti<span class="_ _7"></span>ser<span class="_"> </span>w<span class="_ _1"></span>ould<span class="_"> </span>a&#64256;ect<span class="_ _16"> </span>the</div><div class="t m0 xd h6 y55 ff5 fs3 fc0 sc0 ls0 ws0">strategies<span class="_"> </span>of<span class="_"> </span>other<span class="_"> </span>advertisers.<span class="_ _a"> </span>A<span class="_"> </span>game-theoretical<span class="_"> </span>approac<span class="_ _1"></span>h</div><div class="t m0 xd h6 y56 ff5 fs3 fc0 sc0 ls0 ws0">has<span class="_ _4"> </span>b<span class="_ _12"></span>een<span class="_ _4"> </span>recentl<span class="_ _7"></span>y<span class="_ _4"> </span>prop<span class="_ _12"></span>osed<span class="_ _4"> </span>by<span class="_"> </span>[5].<span class="_ _6"> </span>The<span class="_ _4"> </span>authors<span class="_ _4"> </span>solv<span class="_ _1"></span>ed<span class="_ _9"> </span>R<span class="_ _5"></span>TB</div><div class="t m0 xd h6 y57 ff5 fs3 fc0 sc0 ls0 ws0">with<span class="_ _9"> </span>multi-agen<span class="_ _1"></span>t<span class="_ _2"> </span>reinforcemen<span class="_ _7"></span>t<span class="_ _9"> </span>learning<span class="_ _2"> </span>b<span class="_ _7"></span>y<span class="_ _9"> </span>mo<span class="_ _12"></span>deling<span class="_ _2"> </span>inter-</div><div class="t m0 xd h6 y58 ff5 fs3 fc0 sc0 ls0 ws0">actions<span class="_ _16"> </span>of<span class="_ _f"> </span>bidding<span class="_ _f"> </span>agents.<span class="_ _2"> </span>More<span class="_ _16"> </span>recently<span class="_ _5"></span>,<span class="_"> </span>[3]<span class="_ _16"> </span>formulated<span class="_ _16"> </span>R<span class="_ _5"></span>TB</div><div class="t m0 xd h6 y59 ff5 fs3 fc0 sc0 ls0 ws0">as<span class="_ _b"> </span>a<span class="_ _e"> </span>generalized<span class="_ _b"> </span>mean-&#64257;eld-game<span class="_ _e"> </span>and<span class="_ _b"> </span>prop<span class="_ _12"></span>osed<span class="_ _b"> </span>an<span class="_ _e"> </span>alter-</div><div class="t m0 xd h6 y5a ff5 fs3 fc0 sc0 ls0 ws0">nating<span class="_"> </span>appro<span class="_ _1"></span>ximating<span class="_"> </span>Q-learning<span class="_ _f"> </span>algorithm<span class="_"> </span>with<span class="_"> </span>Boltzmann</div><div class="t m0 xd h6 y5b ff5 fs3 fc0 sc0 ls0 ws0">policy<span class="_ _4"> </span>whic<span class="_ _1"></span>h<span class="_ _4"> </span>demonstrated<span class="_"> </span>superior<span class="_"> </span>numerical<span class="_"> </span>performance.</div><div class="t m0 xd h5 y5c ff4 fs1 fc0 sc0 ls0 ws0">3.<span class="_ _3"> </span>D<span class="_ _1"></span>A<span class="_ _5"></span>T<span class="_ _5"></span>A<span class="_ _4"> </span>EXPLORA<span class="_ _5"></span>TION</div><div class="t m0 xe h6 y5d ff5 fs3 fc0 sc0 ls0 ws0">The<span class="_ _6"> </span>data<span class="_ _e"> </span>set<span class="_ _e"> </span>used<span class="_ _e"> </span>in<span class="_ _6"> </span>this<span class="_ _e"> </span>pap<span class="_ _12"></span>er<span class="_ _e"> </span>is<span class="_ _e"> </span>a<span class="_ _6"> </span>subsample<span class="_ _e"> </span>of<span class="_ _e"> </span>the</div><div class="t m0 xd h6 y5e ff5 fs3 fc0 sc0 ls0 ws0">iPinY<span class="_ _5"></span>ou<span class="_"> </span>logs<span class="_"> </span>[20].<span class="_ _a"> </span>It<span class="_"> </span>only<span class="_"> </span>contains<span class="_"> </span>impressions<span class="_"> </span>a<span class="_ _7"></span>nd<span class="_"> </span>w<span class="_ _7"></span>as<span class="_"> </span>split</div><div class="t m0 xd h6 y5f ff5 fs3 fc0 sc0 ls0 ws0">in<span class="_ _7"></span>to<span class="_"> </span>train/v<span class="_ _1"></span>alidation/test<span class="_"> </span>sets<span class="_ _f"> </span>of<span class="_"> </span>80/10/10<span class="_"> </span>proportions.<span class="_ _a"> </span>Out</div><div class="t m0 xd h6 y60 ff5 fs3 fc0 sc0 ls0 ws0">of<span class="_"> </span>the<span class="_"> </span>25<span class="_"> </span>&#64257;elds<span class="_ _f"> </span>av<span class="_ _1"></span>ailable<span class="_"> </span>for<span class="_ _f"> </span>each<span class="_"> </span>record<span class="_ _7"></span>,<span class="_"> </span>only<span class="_"> </span>22<span class="_"> </span>of<span class="_"> </span>them<span class="_"> </span>c<span class="_ _7"></span>an</div><div class="t m0 xd h6 y61 ff5 fs3 fc0 sc0 ls0 ws0">be<span class="_ _9"> </span>utilized<span class="_ _9"> </span>for<span class="_ _4"> </span>deciding<span class="_ _9"> </span>bid<span class="_ _9"> </span>prices.</div><div class="t m0 xe h6 y62 ff5 fs3 fc0 sc0 ls0 ws0">Purportedly<span class="_ _1"></span>,<span class="_"> </span><span class="ffb">slotprice<span class="_ _f"> </span></span>acts<span class="_"> </span>as<span class="_ _f"> </span>the<span class="_"> </span>minim<span class="_ _1"></span>um<span class="_"> </span>bid<span class="_ _f"> </span>price<span class="_"> </span>for</div><div class="t m0 xd h6 y63 ff5 fs3 fc0 sc0 ls0 ws0">impression.<span class="_ _b"> </span>How<span class="_ _1"></span>ever,<span class="_ _2"> </span>appro<span class="_ _1"></span>ximately<span class="_ _2"> </span>0.5%<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _9"> </span>train<span class="_ _2"> </span>and</div><div class="t m0 xd h6 y64 ff5 fs3 fc0 sc0 ls0 ws0">v<span class="_ _1"></span>alidation<span class="_ _2"> </span>sets<span class="_ _2"> </span>contains<span class="_ _2"> </span>impressions<span class="_ _2"> </span>where<span class="_ _2"> </span>the<span class="_ _2"> </span><span class="ffb">payprice<span class="_ _2"> </span></span>is</div><div class="t m0 xd h6 y65 ff5 fs3 fc0 sc0 ls0 ws0">less<span class="_ _6"> </span>than<span class="_ _6"> </span>the<span class="_ _6"> </span><span class="ffb">slotprice</span>.<span class="_ _10"> </span>This<span class="_ _6"> </span>phenomenon<span class="_ _6"> </span>is<span class="_ _e"> </span>presen<span class="_ _1"></span>t<span class="_ _e"> </span>in</div><div class="t m0 xd h6 y66 ff5 fs3 fc0 sc0 ls0 ws0">both<span class="_ _a"> </span>the<span class="_ _2"> </span>train<span class="_ _a"> </span>and<span class="_ _2"> </span>v<span class="_ _1"></span>alidation<span class="_ _a"> </span>sets.<span class="_ _c"> </span>Notably<span class="_ _5"></span>,<span class="_ _a"> </span>these<span class="_ _a"> </span>impres-</div><div class="t m0 xd h6 y67 ff5 fs3 fc0 sc0 ls0 ws0">sions<span class="_ _a"> </span>ha<span class="_ _1"></span>ve<span class="_ _a"> </span><span class="ff6">av<span class="_ _12"></span>g<span class="_ _12"></span>C<span class="_ _0"></span>T<span class="_ _17"> </span>R</span>s<span class="_ _a"> </span>of<span class="_ _a"> </span>0.06%<span class="_ _a"> </span>whic<span class="_ _1"></span>h<span class="_ _a"> </span>is<span class="_ _a"> </span>comparable<span class="_ _a"> </span>to<span class="_ _a"> </span>the</div><div class="t m0 xd h6 y68 ff5 fs3 fc0 sc0 ls0 ws0">global<span class="_ _4"> </span>a<span class="_ _7"></span>v<span class="_ _7"></span>erage<span class="_ _4"> </span>of<span class="_ _4"> </span>0.07%.<span class="_ _a"> </span>F<span class="_ _1"></span>or<span class="_"> </span>Winning<span class="_ _4"> </span>criterion<span class="_ _4"> </span>#1,<span class="_ _4"> </span>we<span class="_ _4"> </span>can-</div><div class="t m0 xd h6 y69 ff5 fs3 fc0 sc0 ls0 ws0">not<span class="_ _9"> </span>discard<span class="_ _2"> </span>these<span class="_ _2"> </span>impressions<span class="_ _9"> </span>as<span class="_ _2"> </span>winning<span class="_ _9"> </span>them<span class="_ _2"> </span>woul<span class="_ _7"></span>d<span class="_ _2"> </span>yield</div><div class="t m0 xd h6 y6a ff5 fs3 fc0 sc0 ls0 ws0">more<span class="_ _4"> </span>clicks.</div><div class="t m0 xe h6 y6b ff5 fs3 fc0 sc0 ls0 ws0">T<span class="_ _5"></span>able<span class="_ _a"> </span>1<span class="_ _a"> </span>sho<span class="_ _7"></span>ws<span class="_ _a"> </span>the<span class="_ _2"> </span>relev<span class="_ _1"></span>ant<span class="_ _2"> </span>metrics<span class="_ _a"> </span>for<span class="_ _a"> </span>train<span class="_ _a"> </span>and<span class="_ _2"> </span>v<span class="_ _1"></span>alida-</div><div class="t m0 xd h6 y39 ff5 fs3 fc0 sc0 ls0 ws0">tion<span class="_ _a"> </span>sets.<span class="_ _11"> </span>CTR<span class="_ _a"> </span>is<span class="_ _6"> </span>sligh<span class="_ _1"></span>tly<span class="_ _6"> </span>lo<span class="_ _7"></span>w<span class="_ _7"></span>er<span class="_ _a"> </span>in<span class="_ _6"> </span>the<span class="_ _a"> </span>v<span class="_ _1"></span>alidation<span class="_ _a"> </span>set<span class="_ _6"> </span>and</div><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[1.568627,0.000000,0.000000,1.568627,0.000000,0.000000]}'></div></div> </body> </html>
评论
    相关推荐
    • event-reminder-master.zip
      event reminder for remindering event at specific date
    • 人事管理系统(VFP版).rar
      给别人做过的一个毕业设计。用VFP做的。
    • vf图书管理查询系统.rar
      一个典型的图书馆源代码,仅供参考,仅供参考
    • 火车票预定.rar
      火车票预定系统,这是我的课程设计,但由于时间的关系,某些功能还不够完善,但已能完成了基本功能,如查询,预定。
    • jq_gongzi.ARJ
      JQ工资系统
    • 坐标转换的源代码.rar
      VC系统下,地球坐标转换的源代码,在WGS-84坐标和北京54坐标之间的一套转换参数,可以全国通用的,在每个地方会不一样,因为它们是两个不同的椭球基准。
    • jq_caiwu.ARJ
      JQ财务系统
    • 酒店管理系统需求分析 .rar
      随着旅游业的发展,酒店、餐饮娱乐行业日趋发达,引入全方位的电脑服务和电脑管理日益流行。同时,酒店和餐厅娱乐业引入电脑服务和管理也取得了优良的经济效益和社会效益。为此,国家建设部已于最近作出明确规定:凡星级酒店在项目审批时,其设计方案必须包括电脑管理系统,否则不予立项。可见,酒店管理电脑化势在必行。酒店管理系统将先进的电脑技术与现代酒店服务管理完美地结合起来,实现了住宿、餐饮、娱乐全新概念的服务和管理方式。
    • 一个学生管理系统 ACCESS版.rar
      这是一个用D6开发.ACCESS作数据库的一个学生信息管理系统..做得很好....适合毕业设计使用
    • StuDataView.rar
      学生信息管理系统 基于VC++,ODBC,SQL数据库 包含数据库备份文件