[Dynamite] Training -- object model

Wed, 5 Apr 2000 23:48:30 -0700 (PDT)

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

---559023410-851401618-955003710=:4170
Content-Type: TEXT/PLAIN; charset=US-ASCII

Here are some thoughts on the object model of the DP and training
portions of Telegraph, based on what we've already discussed plus some
new stuff. Questions/comments obviously welcome whenever.

Attachments:

  * "Outline.idl" - main interfaces
  * "example.xml" - example XML file

The IDL file gives an overview of the main objects, without methods -
the methods will be outlined below.

The example XML file is there to demonstrate what the XML *might*
*eventually* look like. It also gives a quick overview of the effort
needed to construct a simple model. Try viewing it with tags stripped
out - the emacs command to do this in xml-mode is sgml-tags-invisible.

With regard to the IDL...

Most of the training code lives in ParamModule. The main objects in
ParamModule are Param (a string identifier for e.g. a substitution
matrix or a gap penalty) and Value (a type/value assignment for a
Param). In the example XML, match_to_splice and intron_emit are
Params.

Other modules are MetaParamModule (formerly "MetaSeqModule"; in
"example.xml", splice_donor is a MetaParam) and ModelModule, whose
main objects (Model and Engine) will be described later
below. (Although we plan to do SingleModelModule, PairModelModule and
so on, rather than just ModelModule, most of the design can be
outlined with reference to a hypothetical unified ModelModule as there
are enough shared ideas between the separate XXXModelModules.)

A Param can be thought of as an identifier for a multidimensional
array. A Type encodes the size and dimensions of the array, and a
Subscript is an integer index into a squashed one-dimensional memory
map of the array.

For example, if msc[3]['C'] is the score for seeing a cysteine in the
third match state of a HMMER profile then the appropriate Telegraph
Param would be the string "msc". The Type would be a struct that says
"polymer, single_emit, query_nmer_length=1, query_alphabet=protein"
and the Subscript would be an integer calculated by some formula like
3*20+1 (if 1 is the code for cysteine).

Param names will exist in a single namespace; this means they have to be
qualified, so that e.g. a "match_score" Param belonging to a model
"profile" would actually be called "profile.match_score".

The actual scores for a Param are stored in a Value object which has a
Param, a Type and a multidimensional array. A Param::ValueBuf is a
map (i.e. an associative container) from Params to Values.

A Param::ValueBuf acts as a "bucket" for scores that can be used for
all sorts of things. The main use is to contain all the actual scores
for a particular model, as follows:

Each Transition in a Model corresponds to a single uniquely named
Param (the Model watches out for name clashes and can even auto-assign
sensible Param names for new, unnamed Transitions).

During training, we need to keep track of the number of times each
Param (or Transition) is referenced by a training alignment, in order
to determine the Param's optimal Value.  An alternative use for a
Param::ValueBuf is to store these counts. The counts should be
thought of as living in probability space, but we store them as bit
scores, i.e. as log-counts.

Often having a different Param for each Transition in a model is "too
general" - when we have more Params than actual free variables in the
scoring scheme, e.g.

       * Smith-Waterman
          (more transitions in the model than scores)

       * Profile HMMs
          (different transitions use the same emit vector)

       * GeneWise
          (a GeneWise profile enriches a separate HMMER profile)

       * Other models where transition scores are constrained
          to be identical / related / dependent-on-something-else

       * Null cycle removal from HMMs
          (HMMER's "wing folding" algorithm)

These situations are handled by using an adapter object called a
Param::Mapper (don't worry about all the ::'s - they just translate to
underscores in C, so this is actually a tgParam_Mapper).

A Mapper (for short) can convert a ValueBuf for a 'source' parameter
space {P} (the reduced parameter space, e.g. "gap_open", "gap_extend" &
"subst_matrix" for Smith-Waterman) into a ValueBuf for a 'dest'
space {Q} (the Model Transition Params).

A Mapper can also do the inverse of this problem, which is to
calculate the training counts for {P} given the counts for {Q}.
(These counts are also closely related to the Fisher scores.)

(This is getting fairly deep, but Mappers are basically just
Param-keyed associative containers for ValueFuncs, which are
multidimensional arrays of Funcs (one Func for each entry in a Param's
Value array). An Func is a virtual interface for evaluating a
continuous algebraic function and its first derivatives. The basic
concrete editable Func is ScoreSum (see the Score expressions in
"example.xml" for examples of what ScoreSums can do). Another editable
Func is ProbabilitySum, a probability-space sum of ScoreSum's used for
null cycle removal from HMMs ("wing folding").)

Most users will probably not care about the inner details of Mappers,
but may use them to initialise a Model's Param::ValueBuf. The
garden-variety interface to a Mapper is by the following calls
(remember {P} are the user-friendly parameters, and {Q} are the actual
transition scores for the model):

 tgParam_Mapper_set_Q_zero (tgParam_Mapper mapper,
			    tgParam        Q);

 tgParam_Mapper_add_P_to_Q (tgParam_Mapper mapper,
			    tgParam        P,
			    tgParam        Q);

 tgParam_Mapper_subtract_P_from_Q (tgParam_Mapper mapper,
				   tgParam        P,
			           tgParam        Q);

 tgParam_Mapper_set_Q_neginf (tgParam_Mapper mapper,
			      tgParam        Q);

To build the score expression for the example transition at the
beginning of this text, i.e.

    Transition from poly(Match) to poly(Splice1) {
      ...
      score = match_to_splice + intron_emit(query_i+1);
      ...
    };

you might use something like the following C code (NB tgParam's are
actually GLib Quarks, not strings, so you'd actually have to convert
the string constants using a simple function call first):

    /* ... user_params = {P}, transition_params = {Q} ... */

    tgParam_Mapper mapper = tgParam_new_Mapper (user_params, transition_params);

    tgParam_Mapper_set_Q_zero (mapper, "polyMatch_polySplice1");
    tgParam_Mapper_add_P_to_Q (mapper, "match_to_splice", "polyMatch_polySplice1");
    tgParam_Mapper_add_P_to_Q (mapper, "intron_emit", "polyMatch_polySplice1");

It is perfectly legal to not use a Mapper at all, and set the entries
of a Param::ValueBuf by hand using simple set() calls. The Mapper
just keeps track of the mapping to make it easy to work with the user
parameter space {P} during training (typically, the user variables {P}
will be a more compact, intuitive and training-friendly parameter set
than the model transitions).

Actual DP is done by the Engine which is a bag of virtual algorithms
(including Viterbi-score-only, Viterbi-score-with-alignment,
Forward-backward and Forward-sampling). Things like linear-space
algorithms and code generation should be handled as opaquely as
possible in interfaces derived from Engine.

The DP code in Engine doesn't need to know about Param::Mappers at
all. Mostly it just uses Transitions, Params and Param::ValueBufs.
Pseudocode for a basic Viterbi-score-only implementation is attached.

Training happens mainly in the ParamModule. There are several ways to
train. A Pgroup object is a set of TypedParams that can be used to
normalise a probabilistic group, such as the Params for each
transition leaving a particular state in a Markov model. A
DirichletMix object is a Dirichlet mixture prior over a Pgroup. You
can train with or without Dirichlet priors, over one or more Pgroups,
and there are various methods that do all this.

To train without priors is easiest because the Model automatically
provides a Pgroup::Set with one Pgroup for each state. Each Pgroup
consists of the outgoing transition parameters for the corresponding
state. So, you can put a set of counts in a Param::ValueBuf by one of
the following methods...

      * get the Viterbi traceback using engine.viterbi_traceback, then
        call traceback.add_counts (value_buf)

         OR

      * sample a Forward traceback using engine.sample_traceback, then
        call traceback.add_counts to get the counts

         OR

      * call engine.add_forward_backward_counts (value_buf) to
        estimate counts using the Forward-Backward algorithm

...then call...

	value_buf.normalise_pgroup_set (model.pgroup_set)

...or, if you have a DirichletMixSet prior, call...

        value_buf.train_pgroup_set (model.pgroup_set, prior)

...instead.

Models do *not* have to be probabilistically normalised at any
stage. Forward-backward and other "probabilistic" algorithms treat the
scores as energies and normalise via a partition function (Miyazawa,
Protein Engineering:8, 1994). This reduces to the probabilistic
treatment when the model is pre-normalised.

Global vs local alignment is specified by "tether" attributes in the
model. Local models are allowed to have an "untethering penalty" (so
that the local alignment co-ords can be summed out of the null model
likelihood) but this is not enforced.

Ian

-- 
Ian Holmes  ....  Howard Hughes Medical Institute  ....  ihh@fruitfly.org

---559023410-851401618-955003710=:4170
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="Outline.idl"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.GSO.4.10.10004052348300.4170@fruitfly>
Content-Description: 
Content-Disposition: attachment; filename="Outline.idl"

Ly8gVGVsZWdyYXBoIFBhcmFtcyBhbmQgTW9kZWxzDQoNCm1vZHVsZSBQYXJh
bU1vZHVsZSB7DQoNCiAgdHlwZWRlZiBHUXVhcmsgUGFyYW07ICAvLyB3ZSBz
dG9yZSBzdHJpbmdzIGFzIFF1YXJrcyBjb3MgaXQncyBmYXN0ZXINCiAgY29u
c3QgR1F1YXJrIHVuZGVmaW5lZF9wYXJhbSA9ICIiOw0KICBleGNlcHRpb24g
VW5kZWZpbmVkOw0KICBleGNlcHRpb24gTm90Rm91bmQ7DQoNCiAgdHlwZWRl
ZiB1bnNpZ25lZCBsb25nIFN1YnNjcmlwdDsNCiAgZXhjZXB0aW9uIFN1YnNj
cmlwdE91dE9mUmFuZ2U7DQoNCiAgaW50ZXJmYWNlIFR5cGUgeyAgLy8gQWxw
aGFiZXRzICsgbm1lci1sZW5ndGhzICsgcG9seW1lciBzaXplLCBnZW5lcmF0
ZXMgU3VzYmNyaXB0cw0KICAgIGV4Y2VwdGlvbiBNaXNtYXRjaDsNCiAgfTsN
Cg0KICBpbnRlcmZhY2UgVmFsdWU7ICAvLyBQYXJhbSArIFR5cGUgKyBTdWJz
Y3JpcHQtaW5kZXhlZCBTY29yZVZlY3Rvcg0KICBpbnRlcmZhY2UgVmFsdWVC
dWY7ICAvLyBtYXBwaW5nIGZyb20gUGFyYW0gdG8gVmFsdWUgd2l0aCBQZ3Jv
dXAgdHJhaW5pbmcgbWV0aG9kcw0KICANCiAgaW50ZXJmYWNlIEZ1bmMgeyAg
Ly8gZGlmZmVyZW50aWFibGUgc2NvcmUgZnVuY3Rpb24NCiAgICBpbnRlcmZh
Y2UgU2NvcmVTdW0gOiBGdW5jOyAgLy8gYSBzdW0gb2YgUGFyYW0gc2NvcmVz
DQogICAgaW50ZXJmYWNlIFByb2JhYmlsaXR5U3VtIDogRnVuYzsgIC8vIGEg
c3VtIG9mIFBhcmFtIHByb2JhYmlsaXRpZXMNCiAgfTsNCg0KICBpbnRlcmZh
Y2UgVmFsdWVGdW5jOyAgLy8gUGFyYW0gKyBUeXBlICsgU3Vic2NyaXB0LWlu
ZGV4ZWQgRnVuY1ZlY3Rvcg0KICBpbnRlcmZhY2UgTWFwcGVyOyAgLy8gY29u
dmVydHMgc2NvcmVzIGFuZCB0cmFpbmluZyBjb3VudHMgKG1hcHMgUGFyYW0g
dG8gVmFsdWVGdW5jKQ0KICANCiAgaW50ZXJmYWNlIFBncm91cDsgIC8vIGdy
b3VwIG9mIG11dHVhbGx5IGV4Y2x1c2l2ZSBQYXJhbXMgb2YgY29tcGF0aWJs
ZSBUeXBlDQogIGludGVyZmFjZSBEaXJpY2hsZXRNaXg7ICAvLyB3ZWlnaHRl
ZCBtaXh0dXJlIG9mIERpcmljaGxldHMgZm9yIGEgUGdyb3VwLCBzdG9yZWQg
aW4gVmFsdWU6OkJ1ZnMNCg0KICBpbnRlcmZhY2UgUGdyb3VwU2V0OyAgLy8g
bGlzdCBvZiBQZ3JvdXBzIChlLmcuIHRoZSBvdXRnb2luZyB0cmFuc2l0aW9u
cyBmb3IgZWFjaCBTdGF0ZSBpbiBhIE1vZGVsKQ0KICBpbnRlcmZhY2UgRGly
aWNobGV0TWl4U2V0OyAgLy8gbGlzdCBvZiBQZ3JvdXA6OkRpcmljaGxldE1p
eCdzIGZvciBlYWNoIFBncm91cCBpbiBhIFBncm91cDo6U2V0DQogIA0KICBp
bnRlcmZhY2UgTnVsbE1vZGVsOyAgLy8gYW4gZW1pdCB2ZWN0b3Igd2l0aCBl
eHRlbmQgYW5kIGVuZCBzY29yZXMgdGhhdCBwcm92aWRlIGEgbGVuZ3RoIGRp
c3RyaWJ1dGlvbiBvdmVyIHNlcXVlbmNlcw0KDQogIGludGVyZmFjZSBGYWN0
b3J5OyAgLy8gY29uc3RydWN0b3JzIGZvciBhbGwgaW50ZXJmYWNlcw0KICAN
Cn07DQoNCm1vZHVsZSBNZXRhUGFyYW1Nb2R1bGUgew0KDQogIHR5cGVkZWYg
R1F1YXJrIE1ldGFQYXJhbTsNCiAgdHlwZWRlZiBHUXVhcmsgTGFiZWw7DQoN
CiAgaW50ZXJmYWNlIFNjb3JlU2VxOyAgLy8gU2NvcmU6OlNjb3JlVmVjdG9y
IHdyYXBwZXINCiAgaW50ZXJmYWNlIE1ldGFTZXE7ICAvLyBtYXAgZnJvbSBN
ZXRhUGFyYW1zIHRvIFNjb3JlU2Vxcw0KDQogIGludGVyZmFjZSBNZXRhR3Jv
dXA7ICAvLyBncm91cCBvZiBtdXR1YWxseSBleGNsdXNpdmUgTWV0YVBhcmFt
cyAoZS5nLiAic3BsaWNlX3NpdGUiIHZzICJub3Rfc3BsaWNlX3NpdGUiKQ0K
ICBpbnRlcmZhY2UgTWV0YUdyb3VwU2V0OyAgLy8gbGlzdCBvZiBNUGdyb3Vw
cw0KDQogIGludGVyZmFjZSBPZmZzZXRTY29yZVN1bTsgIC8vIHN1bSBvZiBN
ZXRhUGFyYW1zIHdpdGggbG9va2JhY2sgb2Zmc2V0cyBmb3IgYSBMaWdodFNl
cQ0KDQogIGludGVyZmFjZSBNZXRhU2VxRGF0YWJhc2U7ICAgLy8gdmlydHVh
bCBmYWN0b3J5IGZvciBtYWtpbmcgTWV0YVNlcXMgZnJvbSBTZXFzDQogIGlu
dGVyZmFjZSBGYWN0b3J5OyAgLy8gY29uc3RydWN0b3JzDQp9Ow0KDQoNCm1v
ZHVsZSBNb2RlbE1vZHVsZSB7ICAgLy8gb3VyIHBsYW4gaXMgdG8gaGF2ZSBT
aW5nbGVNb2RlbE1vZHVsZSwgUGFpck1vZGVsTW9kdWxlIGV0YyBldGMsIGJ1
dCB0aGV5J2xsIGFsbCBsb29rIGEgYml0IGxpa2UgdGhpcw0KICANCiAgc3Ry
dWN0IFN0YXRlOyAgLy8gbmFtZSArIHBvbHltZXIgZmxhZw0KICBzdHJ1Y3Qg
VHJhbnNpdGlvbjsgIC8vIGZyb20tU3RhdGUgKyB0by1TdGF0ZSArIG5tZXIt
bGVuZ3Rocw0KDQogIGludGVyZmFjZSBNb2RlbCB7ICAvLyB2aXJ0dWFsIHRh
YmxlIG9mIChUcmFuc2l0aW9uLFBhcmFtKSArIHNlcXVlbmNlPFN0YXRlPiAr
IG5hbWUNCiAgICBpbnRlcmZhY2UgQnVmIDogTW9kZWw7ICAvLyB3cml0ZWFi
bGUgTW9kZWwNCiAgfTsNCg0KICBpbnRlcmZhY2UgVHJhY2ViYWNrOyAgLy8g
c2VxdWVuY2U8VHJhbnNpdGlvbj4gKyBiaW9zZXF1ZW5jZSBzdGFydCBpbmRl
eCANCg0KICBpbnRlcmZhY2UgRW5naW5lIHsgIC8vIGJhZyBvZiBhbGdvcml0
aG1zIGZvciBhIE1vZGVsDQogICAgaW50ZXJmYWNlIFJlc2VhcmNoIDogRW5n
aW5lOyAgLy8gcmVzZWFyY2ggaW1wbGVtZW50YXRpb25zLCBjb2RlZCBmb3Ig
cmVhZGFiaWxpdHkgYXMgd2VsbCBhcyBzcGVlZA0KICAgIGludGVyZmFjZSBD
b2RlTWFuYWdlciA6IEVuZ2luZTsgIC8vIGhhbmRsZXMgbG9jYWwgY29kZSBn
ZW5lcmF0aW9uDQogIH07DQp9Ow0K
---559023410-851401618-955003710=:4170
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="example.xml"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.GSO.4.10.10004052348301.4170@fruitfly>
Content-Description: 
Content-Disposition: attachment; filename="example.xml"

PHRlbGVncmFwaC1tb2RlbD5Nb2RlbDogPG1vZGVsLW5hbWU+dW5nYXBwZWRf
aW50cm9uX21hdGNoZXI8L21vZGVsLW5hbWU+DQoNCjxhbHBoYWJldD4NCk5l
dyBhbHBoYWJldDogPG5hbWU+RE5BPC9uYW1lPg0KQ2hhcmFjdGVyczogPGNo
YXJzPmFjZ3Q8L2NoYXJzPg0KPGRlZ2VuZXJhY3ktdGFibGU+RGVnZW5lcmFj
eSB0YWJsZToNCiAgICA8cm93PiA8Y2hhcj5yPC9jaGFyPiA9IDxjaGFycz5h
ZzwvY2hhcnM+IDwvcm93Pg0KICAgIDxyb3c+IDxjaGFyPnk8L2NoYXI+ID0g
PGNoYXJzPmN0PC9jaGFycz4gPC9yb3c+DQogICAgPHJvdz4gPGNoYXI+bjwv
Y2hhcj4gPSA8Y2hhcnM+YWNndDwvY2hhcnM+IDwvcm93Pg0KICAgIDwhLS0g
ZXRjLCBldGMgLS0+DQo8L2RlZ2VuZXJhY3ktdGFibGU+PC9hbHBoYWJldD4N
Cg0KPHNlcS1kZWNsPlNlcXVlbmNlczogPHNlcT48bmFtZT5RdWVyeTwvbmFt
ZT4gKDxhbHBoYWJldD5ETkE8L2FscGhhYmV0Pik8L3NlcT48L3NlcS1kZWNs
Pg0KDQo8cGFyYW0tZGVjbD4NCjxwZ3JvdXAtc2V0PlByb2JhYmlsaXN0aWMg
cGFyYW1ldGVyIGdyb3Vwcw0KICAgIDxudWxsLWVtaXQ+PHBncm91cD4gR3Jv
dXAgIzxudW0+MDwvbnVtPiAobnVsbCBlbWl0IGdyb3VwKQ0KICAgICAgICA8
c2luZ2xlPiBTaW5nbGUtZW1pdCAoPHNpemU+MTwvc2l6ZT4qPGFscGhhYmV0
PkROQTwvYWxwaGFiZXQ+KTogPG5hbWU+bnVsbF9lbWl0PC9uYW1lPiA8L3Np
bmdsZS1lbWl0Pg0KICAgIDwvcGdyb3VwPjwvbnVsbC1lbWl0Pg0KDQogICAg
PHBncm91cD4gR3JvdXAgIzxudW0+MTwvbnVtPg0KICAgICAgICA8c2luZ2xl
PiBTaW5nbGUtZW1pdCAoPHNpemU+MTwvc2l6ZT4qPGFscGhhYmV0PkROQTwv
YWxwaGFiZXQ+KTogPG5hbWU+aW50cm9uX2VtaXQ8L25hbWU+IDwvc2luZ2xl
LWVtaXQ+DQogICAgPC9wZ3JvdXA+DQoNCiAgICA8cGdyb3VwPiBHcm91cCAj
PG51bT4yPC9udW0+DQogICAgICAgIDxwb2x5LXNpbmdsZT4gUG9seW1lci1z
aW5nbGUtZW1pdCAoPHNpemU+MTwvc2l6ZT4qPGFscGhhYmV0PkROQTwvYWxw
aGFiZXQ+KTogPG5hbWU+bWF0Y2hfZW1pdDwvbmFtZT4gPC9wb2x5LXNpbmds
ZT4NCiAgICA8L3Bncm91cD4NCg0KICAgIDxwZ3JvdXA+IEdyb3VwICM8bnVt
PjM8L251bT4NCiAgICAgICAgPHNpbGVudD4gU2lsZW50OiA8bmFtZT5tYXRj
aF90b19pbnRyb248L25hbWU+LCA8bmFtZT5tYXRjaF90b19tYXRjaDwvbmFt
ZT4gPC9wb2x5LXNpbmdsZT4NCiAgICA8L3Bncm91cD4NCg0KICAgIDxwZ3Jv
dXA+IEdyb3VwICM8bnVtPjQ8L251bT4NCiAgICAgICAgPHNpbGVudD4gU2ls
ZW50OiA8bmFtZT5pbnRyb25fdG9faW50cm9uPC9uYW1lPiwgPG5hbWU+aW50
cm9uX3RvX21hdGNoPC9uYW1lPiA8L3BvbHktc2luZ2xlPg0KICAgIDwvcGdy
b3VwPg0KDQogICAgPHBncm91cD4gR3JvdXAgIzxudW0+NTwvbnVtPg0KICAg
ICAgICA8cG9seS1zaWxlbnQ+IFBvbHltZXItc2lsZW50OiA8bmFtZT5zdGFy
dF90b19tYXRjaDwvbmFtZT4gPC9wb2x5LXNpbGVudD4NCgk8c2lsZW50PiBT
aWxlbnQ6IDxuYW1lPnN0YXJ0X3RvX2VuZDwvbmFtZT4gPC9zaWxlbnQ+DQog
ICAgPC9wZ3JvdXA+DQoNCiAgICA8cGdyb3VwPiBHcm91cCAjPG51bT42PC9u
dW0+DQogICAgICAgIDxwb2x5LXNpbGVudD4gUG9seW1lci1zaWxlbnQ6IDxu
YW1lPm1hdGNoX3RvX2VuZDwvbmFtZT4gPC9wb2x5LXNpbGVudD4NCiAgICA8
L3Bncm91cD4NCjwvcGdyb3VwLXNldD4NCiAgICANCjxtZXRhZ3JvdXAtc2V0
PlByb2JhYmlsaXN0aWMgbWV0YXBhcmFtZXRlciBncm91cHMNCiAgICA8bWV0
YWdyb3VwPiBHcm91cCAjPG51bT4xPC9udW0+Og0KICAgICAgICA8bnVsbD48
bmFtZT5ub19zcGxpY2VfZG9ub3I8L25hbWU+KG51bGwpPC9udWxsPiwgPG5h
bWU+c3BsaWNlX2Rvbm9yPC9uYW1lPg0KICAgIDwvbWV0YWdyb3VwPg0KDQog
ICAgPG1ldGFncm91cD4gR3JvdXAgIzxudW0+MTwvbnVtPjoNCiAgICAgICAg
PG51bGw+PG5hbWU+bm9fc3BsaWNlX2FjY2VwdG9yPC9uYW1lPihudWxsKTwv
bnVsbD4sIDxuYW1lPnNwbGljZV9hY2NlcHRvcjwvbmFtZT4NCiAgICA8L21l
dGFncm91cD4NCjwvbWV0YWdyb3VwLXNldD48L3BhcmFtLWRlY2w+DQogICAg
DQo8c3RhdGUtZGVjbD5TdGF0ZSBkZWNsYXJhdGlvbnMNCihTdGFydCBhbmQg
RW5kIHN0YXRlcyBhcmUgYXV0b21hdGljKQ0KPHBvbHktc3RhdGU+IFBvbHlt
ZXIgc3RhdGVzOiA8bmFtZT5NYXRjaDwvbmFtZT4gPG5hbWU+SW50cm9uPC9u
YW1lPiA8L3BvbHktc3RhdGU+DQo8L3N0YXRlLWRlY2w+DQoNCg0KPHRyYW5z
aXRpb24tZGVjbD5UcmFuc2l0aW9uIGRlY2xhcmF0aW9ucw0KICAgIDxwb2x5
LXRvLXBvbHktc2luZ2xlLXRyYW5zPg0KICAgIE5ldyB0cmFuc2l0aW9uIGZy
b20gcG9seS08ZnJvbT5NYXRjaDwvZnJvbT5bbl0gdG8gcG9seS08dG8+TWF0
Y2g8L3RvPltuPHBvbHktb2Zmc2V0PisxPC9wb2x5LW9mZnNldD5dIGVtaXRz
IDxlbWl0PjxzaXplPjE8L3NpemU+IHJlc2lkdWUgc3RhcnRpbmcgYXQgPHNl
cT5RdWVyeTwvc2VxPltpXTwvZW1pdD4NCiAgICBMYWJlbDogPGxhYmVsPk1h
dGNoPC9sYWJlbD4NCiAgICA8Y2FsYz5TY29yZSA9IDxwbHVzLXNpbGVudD4r
IDxuYW1lPm1hdGNoX3RvX21hdGNoPC9uYW1lPjwvcGx1cy1zaWxlbnQ+DQoJ
ICAgIDxwbHVzLXBvbHktZW1pdD4rIDxuYW1lPm1hdGNoX2VtaXQ8L25hbWU+
IFtuPHBvbHktb2Zmc2V0PiswXSBbPHNlcT5RdWVyeTwvc2VxPiBbaSs8c2Vx
b2Zmc2V0PjA8L3NlcW9mZnNldD5dXTwvcGx1cy1wb2x5LWVtaXQ+DQoJICAg
IDxtaW51cy1lbWl0Pi0gPG5hbWU+bnVsbF9lbWl0PC9uYW1lPiBbPHNlcT5R
dWVyeTwvc2VxPiBbaTxzZXFvZmZzZXQ+KzA8L3NlcW9mZnNldD5dXTwvbWlu
dXMtZW1pdD4NCiAgICA8L2NhbGM+DQogICAgPC9wb2x5LXRvLXBvbHktc2lu
Z2xlLXRyYW5zPg0KDQogICAgPHBvbHktdG8tcG9seS1zaW5nbGUtdHJhbnM+
DQogICAgTmV3IHRyYW5zaXRpb24gZnJvbSBwb2x5LTxmcm9tPk1hdGNoPC9m
cm9tPltuXSB0byBwb2x5LTx0bz5JbnRyb248L3RvPltuPHBvbHktb2Zmc2V0
PiswPC9wb2x5LW9mZnNldD5dIGVtaXRzIDxlbWl0PjxzaXplPjE8L3NpemU+
IHJlc2lkdWUgc3RhcnRpbmcgYXQgPHNlcT5RdWVyeTwvc2VxPltpXTwvZW1p
dD4NCiAgICBMYWJlbDogPGxhYmVsPjUnIHNwbGljZTwvbGFiZWw+DQogICAg
PGNhbGM+U2NvcmUgPSA8cGx1cy1zaWxlbnQ+KyA8bmFtZT5tYXRjaF90b19p
bnRyb248L25hbWU+PC9wbHVzLXNpbGVudD4NCgkgICAgPHBsdXMtZW1pdD4r
IDxuYW1lPmludHJvbl9lbWl0PC9uYW1lPiBbPHNlcT5RdWVyeTwvc2VxPiBb
aTxzZXFvZmZzZXQ+KzA8L3NlcW9mZnNldD5dXTwvcGx1cy1lbWl0Pg0KCSAg
ICA8bWludXMtZW1pdD4tIDxuYW1lPm51bGxfZW1pdDwvbmFtZT4gWzxzZXE+
UXVlcnk8L3NlcT4gW2k8c2Vxb2Zmc2V0PiswPC9zZXFvZmZzZXQ+XV08L21p
bnVzLWVtaXQ+DQoJICAgIDxwbHVzLW1ldGE+KyA8c2VxPlF1ZXJ5PC9zZXE+
LjxuYW1lPnNwbGljZV9kb25vcjwvbmFtZT4gW2k8c2Vxb2Zmc2V0PiswPC9z
ZXFvZmZzZXQ+XTwvcGx1cy1tZXRhPg0KCSAgICA8bWludXMtbWV0YT4tIDxz
ZXE+UXVlcnk8L3NlcT4uPG5hbWU+bm9fc3BsaWNlX2Rvbm9yPC9uYW1lPiBb
aTxzZXFvZmZzZXQ+KzA8L3NlcW9mZnNldD5dPC9taW51cy1tZXRhPg0KICAg
IDwvY2FsYz4NCiAgICA8L3BvbHktdG8tcG9seS1zaW5nbGUtdHJhbnM+DQoN
CiAgICA8cG9seS10by1wb2x5LXNpbmdsZS10cmFucz4NCiAgICBOZXcgdHJh
bnNpdGlvbiBmcm9tIHBvbHktPGZyb20+SW50cm9uPC9mcm9tPltuXSB0byBw
b2x5LTx0bz5JbnRyb248L3RvPltuPHBvbHktb2Zmc2V0PiswPC9wb2x5LW9m
ZnNldD5dIGVtaXRzIDxlbWl0PjxzaXplPjE8L3NpemU+IHJlc2lkdWUgc3Rh
cnRpbmcgYXQgPHNlcT5RdWVyeTwvc2VxPltpXTwvZW1pdD4NCiAgICBMYWJl
bDogPGxhYmVsPkludHJvbjwvbGFiZWw+DQogICAgPGNhbGM+U2NvcmUgPSA8
cGx1cy1zaWxlbnQ+KyA8bmFtZT5pbnRyb25fdG9faW50cm9uPC9uYW1lPjwv
cGx1cy1zaWxlbnQ+DQoJICAgIDxwbHVzLWVtaXQ+KyA8bmFtZT5pbnRyb25f
ZW1pdDwvbmFtZT4gWzxzZXE+UXVlcnk8L3NlcT4gW2k8c2Vxb2Zmc2V0Pisw
PC9zZXFvZmZzZXQ+XV08L3BsdXMtZW1pdD4NCgkgICAgPG1pbnVzLWVtaXQ+
LSA8bmFtZT5udWxsX2VtaXQ8L25hbWU+IFs8c2VxPlF1ZXJ5PC9zZXE+IFtp
PHNlcW9mZnNldD4rMDwvc2Vxb2Zmc2V0Pl1dPC9taW51cy1lbWl0Pg0KICAg
IDwvY2FsYz48L3BvbHktdG8tcG9seS1zaW5nbGUtdHJhbnM+DQoNCiAgICA8
cG9seS10by1wb2x5LXNpbmdsZS10cmFucz4NCiAgICBOZXcgdHJhbnNpdGlv
biBmcm9tIHBvbHktPGZyb20+SW50cm9uPC9mcm9tPltuXSB0byBwb2x5LTx0
bz5NYXRjaDwvdG8+W248cG9seS1vZmZzZXQ+KzE8L3BvbHktb2Zmc2V0Pl0g
ZW1pdHMgPGVtaXQ+PHNpemU+MTwvc2l6ZT4gcmVzaWR1ZSBzdGFydGluZyBh
dCA8c2VxPlF1ZXJ5PC9zZXE+W2ldPC9lbWl0Pg0KICAgIExhYmVsOiA8bGFi
ZWw+Mycgc3BsaWNlPC9sYWJlbD4NCiAgICA8Y2FsYz5TY29yZSA9IDxwbHVz
LXNpbGVudD4rIDxuYW1lPmludHJvbl90b19tYXRjaDwvbmFtZT48L3BsdXMt
c2lsZW50Pg0KCSAgICA8cGx1cy1wb2x5LWVtaXQ+KyA8bmFtZT5tYXRjaF9l
bWl0PC9uYW1lPiBbbjxwb2x5LW9mZnNldD4rMF0gWzxzZXE+UXVlcnk8L3Nl
cT4gW2krPHNlcW9mZnNldD4wPC9zZXFvZmZzZXQ+XV08L3BsdXMtcG9seS1l
bWl0Pg0KCSAgICA8bWludXMtZW1pdD4tIDxuYW1lPm51bGxfZW1pdDwvbmFt
ZT4gWzxzZXE+UXVlcnk8L3NlcT4gW2k8c2Vxb2Zmc2V0PiswPC9zZXFvZmZz
ZXQ+XV08L21pbnVzLWVtaXQ+DQoJICAgIDxwbHVzLW1ldGE+KyA8c2VxPlF1
ZXJ5PC9zZXE+LjxuYW1lPnNwbGljZV9hY2NlcHRvcjwvbmFtZT4gW2k8c2Vx
b2Zmc2V0PiswPC9zZXFvZmZzZXQ+XTwvcGx1cy1tZXRhPg0KCSAgICA8bWlu
dXMtbWV0YT4tIDxzZXE+UXVlcnk8L3NlcT4uPG5hbWU+bm9fc3BsaWNlX2Fj
Y2VwdG9yPC9uYW1lPiBbaTxzZXFvZmZzZXQ+KzA8L3NlcW9mZnNldD5dPC9t
aW51cy1tZXRhPg0KICAgIDwvY2FsYz48L3BvbHktdG8tcG9seS1zaW5nbGUt
dHJhbnM+DQoNCiAgICA8bW9uby10by1wb2x5LXNpbmdsZS10cmFucz4NCiAg
ICBOZXcgdHJhbnNpdGlvbiBmcm9tIDxmcm9tPlN0YXJ0PC9mcm9tPiB0byBw
b2x5LTx0bz5NYXRjaDwvdG8+W25dIGVtaXRzIDxlbWl0PjxzaXplPjE8L3Np
emU+IHJlc2lkdWUgc3RhcnRpbmcgYXQgPHNlcT5RdWVyeTwvc2VxPltpXTwv
ZW1pdD4NCiAgICBMYWJlbDogPGxhYmVsPkJlZ2luL01hdGNoPC9sYWJlbD4N
CiAgICA8Y2FsYz5TY29yZSA9IDxwbHVzLXBvbHktc2lsZW50PisgPG5hbWU+
c3RhcnRfdG9fbWF0Y2g8L25hbWU+IFtuPHBvbHktb2Zmc2V0PiswPC9wb2x5
LW9mZnNldD5dPC9wbHVzLXBvbHktc2lsZW50Pg0KCSAgICA8cGx1cy1wb2x5
LWVtaXQ+KyA8bmFtZT5tYXRjaF9lbWl0PC9uYW1lPiBbbjxwb2x5LW9mZnNl
dD4rMF0gWzxzZXE+UXVlcnk8L3NlcT4gW2krPHNlcW9mZnNldD4wPC9zZXFv
ZmZzZXQ+XV08L3BsdXMtcG9seS1lbWl0Pg0KCSAgICA8bWludXMtZW1pdD4t
IDxuYW1lPm51bGxfZW1pdDwvbmFtZT4gWzxzZXE+UXVlcnk8L3NlcT4gW2k8
c2Vxb2Zmc2V0PiswPC9zZXFvZmZzZXQ+XV08L21pbnVzLWVtaXQ+DQogICAg
PC9jYWxjPjwvbW9uby10by1wb2x5LXNpbmdsZS10cmFucz4NCg0KICAgIDxw
b2x5LXRvLW1vbm8tc2lsZW50LXRyYW5zPg0KICAgIE5ldyB0cmFuc2l0aW9u
IGZyb20gcG9seS08ZnJvbT5NYXRjaDwvZnJvbT5bbl0gdG8gPHRvPkVuZDwv
dG8+DQogICAgTGFiZWw6IDxsYWJlbD5FbmQ8L2xhYmVsPg0KICAgIDxjYWxj
PlNjb3JlID0gPHBsdXMtcG9seS1zaWxlbnQ+KyA8bmFtZT5tYXRjaF90b19l
bmQ8L25hbWU+IFtuPHBvbHktb2Zmc2V0PiswPC9wb2x5LW9mZnNldD5dPC9w
bHVzLXBvbHktc2lsZW50Pg0KICAgIDwvY2FsYz48L3BvbHktdG8tbW9uby1z
aWxlbnQtdHJhbnM+DQoNCiAgICA8bW9uby10by1tb25vLXNpbGVudC10cmFu
cz4NCiAgICBOZXcgdHJhbnNpdGlvbiBmcm9tIDxmcm9tPlN0YXJ0PC9mcm9t
PiB0byA8dG8+RW5kPC90bz4NCiAgICBMYWJlbDogPGxhYmVsPk51bGwgYWxp
Z25tZW50PC9sYWJlbD4NCiAgICA8Y2FsYz5TY29yZSA9IDxwbHVzLXNpbGVu
dD4rIDxuYW1lPnN0YXJ0X3RvX2VuZDwvbmFtZT4gPC9wbHVzLXNpbGVudD4N
CiAgICA8L2NhbGM+PC9tb25vLXRvLW1vbm8tc2lsZW50LXRyYW5zPg0KDQo8
L3RyYW5zaXRpb24tZGVjbD4NCg0KPHBhcmFtLXNjb3Jlcz48bmFtZT5FeGFt
cGxlPC9uYW1lPiBzY29yZXMsIHBvbHltZXIgc2l6ZSA9IDxwb2x5LXNpemU+
NTwvcG9seS1zaXplPg0KICAgIDxzaW5nbGUtcGFyYW0tc2NvcmU+PHBhcmFt
PmludHJvbl9lbWl0PC9wYXJhbT4gPSB7IDxzY29yZT4tMTIzMiwgLTQ1LCAt
NzkzMiwgLTEwMzIxPC9zY29yZT4gfSBiaXRzPC9zaW5nbGUtcGFyYW0tc2Nv
cmU+DQogICAgPHNpbGVudC1wYXJhbS1zY29yZT48cGFyYW0+bWF0Y2hfdG9f
c3BsaWNlPC9wYXJhbT4gPSA8c2NvcmU+LTEzNDk8L3Njb3JlPiBiaXRzPC9z
aWxlbnQtcGFyYW0tc2NvcmU+DQo8L3BhcmFtLXNjb3Jlcz4NCg0KTWV0YXBh
cmFtIHNoZWxsIGNvbW1hbmQ6IDxtZXRhLXBhcmFtLWNvbW1hbmQ+bXlfc3Bs
aWNlX3ByZWRpY3Rlci5wbDwvbWV0YS1wYXJhbS1jb21tYW5kPg0KDQo8L3Rl
bGVncmFwaC1tb2RlbD4NCg==
---559023410-851401618-955003710=:4170--