3 \m@sdZdZddlZddlZddlmZdZiaddZGdd d e Z Gd d d e Z Gd d d e Z ddZ Gddde ZGdddeZGdddeZGdddeZGdddeZddZdS)z Python parse tree definitions. This is a very concrete parse tree; we need to keep every token and even the comments and whitespace between tokens. There's also a pattern matching implementation here. z#Guido van Rossum N)StringIOicCsHtst|jjD].\}}||kr|jj|jj|=d|_|SqWdS)z Remove the node from the tree. Returns the position of the node in its parent's children before it was removed. N)r! enumerater%r()rir0r r rremoves  z Base.removec CsZ|jdkrdSxFt|jjD]6\}}||kry|jj|dStk rPdSXqWdS)z The node immediately following the invocant in their parent's children list. If the invocant does not have a next sibling, it is None Nr)r!r3r% IndexError)rr4childr r r next_siblings zBase.next_siblingcCsP|jdkrdSxsys version_inforBr r r rr s0       rc@seZdZdZdddZddZddZejdkr4eZ d d Z d dZ ddZ ddZ ddZddZeeeZddZddZddZdS)Nodez+Concrete implementation for interior nodes.NcCsx|dkst|||_t||_x*|jD] }|jdksBtt|||_q(W|dk rZ||_|rn|dd|_nd|_dS)z Initializer. Takes a type constant (a symbol number >= 256), a sequence of child nodes, and an optional context keyword argument. As a side effect, the parent pointers of the children are updated. N)rrr$r%r!reprr=fixers_applied)rrr%contextr=rNr+r r r__init__s    z Node.__init__cCsd|jjt|j|jfS)z)Return a canonical string representation.z %s(%s, %r))rrCrrr%)rr r r__repr__sz Node.__repr__cCsdjtt|jS)zk Return a pretty string representation. This reproduces the input source exactly. r<)joinmapr"r%)rr r r __unicode__szNode.__unicode__r?rcCs|j|jf|j|jfkS)zCompare two nodes for equality.)rr%)rrr r rrszNode._eqcCst|jdd|jD|jdS)z$Return a cloned (deep) copy of self.cSsg|] }|jqSr )r).0r+r r r szNode.clone..)rN)rKrr%rN)rr r rrsz Node.cloneccs(x|jD]}|jEdHqW|VdS)z*Return a post-order iterator for the tree.N)r%r)rr7r r rrs zNode.post_orderccs(|Vx|jD]}|jEdHqWdS)z)Return a pre-order iterator for the tree.N)r%r )rr7r r rr s zNode.pre_ordercCs|js dS|jdjS)zO The whitespace and comments preceding this node in the input. r<r)r%r=)rr r r_prefix_getterszNode._prefix_gettercCs|jr||jd_dS)Nr)r%r=)rr=r r r_prefix_setterszNode._prefix_settercCs(||_d|j|_||j|<|jdS)z Equivalent to 'node.children[i] = child'. This method also sets the child's parent attribute appropriately. N)r!r%r()rr4r7r r r set_child!s  zNode.set_childcCs ||_|jj|||jdS)z Equivalent to 'node.children.insert(i, child)'. This method also sets the child's parent attribute appropriately. N)r!r%insertr()rr4r7r r r insert_child+szNode.insert_childcCs||_|jj||jdS)z Equivalent to 'node.children.append(child)'. This method also sets the child's parent attribute appropriately. N)r!r%r'r()rr7r r r append_child4s zNode.append_child)NNN)r?r)rCrDrErFrPrQrTrIrJrBrrrr rWrXrHr=rYr[r\r r r rrKs$     rKc@seZdZdZdZdZdZddgfddZddZd d Z e j dkrFe Z d d Z ddZddZddZddZddZddZeeeZdS)r.z'Concrete implementation for leaf nodes.r<rNcCsfd|kodkns t||dk r:|\|_\|_|_||_||_|dk rT||_|dd|_dS)z Initializer. Takes a type constant (a token number < 256), a string value, and an optional context keyword argument. rrLN)r_prefixr/columnrvaluerN)rrr_rOr=rNr r rrPGs z Leaf.__init__cCsd|jj|j|jfS)z)Return a canonical string representation.z %s(%r, %r))rrCrr_)rr r rrQZsz Leaf.__repr__cCs|jt|jS)zk Return a pretty string representation. This reproduces the input source exactly. )r=r"r_)rr r rrT`szLeaf.__unicode__r?cCs|j|jf|j|jfkS)zCompare two nodes for equality.)rr_)rrr r rrkszLeaf._eqcCs$t|j|j|j|j|jff|jdS)z$Return a cloned (deep) copy of self.)rN)r.rr_r=r/r^rN)rr r rros z Leaf.cloneccs |VdS)Nr )rr r rr:usz Leaf.leavesccs |VdS)z*Return a post-order iterator for the tree.Nr )rr r rrxszLeaf.post_orderccs |VdS)z)Return a pre-order iterator for the tree.Nr )rr r rr |szLeaf.pre_ordercCs|jS)zP The whitespace and comments preceding this token in the input. )r])rr r rrWszLeaf._prefix_gettercCs|j||_dS)N)r(r])rr=r r rrXszLeaf._prefix_setter)r?r)rCrDrErFr]r/r^rPrQrTrIrJrBrrr:rr rWrXrHr=r r r rr.>s&  r.cCsN|\}}}}|s||jkrConstructor that prevents BasePattern from being instantiated.zCannot instantiate BasePattern)rbrrr)rrrr r rrszBasePattern.__new__cCsLt|j|j|jg}x|r.|ddkr.|d=qWd|jjdjtt|fS)Nrz%s(%s)z, rc) rrcontentr rrCrRrSrM)rrr r rrQs zBasePattern.__repr__cCs|S)z A subclass can define this as a hook for optimizations. Returns either self or another node with the same effect. r )rr r roptimizeszBasePattern.optimizecCsn|jdk r|j|jkrdS|jdk rRd}|dk r4i}|j||sDdS|rR|j||dk rj|jrj|||j<dS)a# Does this pattern exactly match a node? Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. Default implementation for non-wildcard patterns. NFT)rrd _submatchupdater )rr0resultsrr r rmatchs     zBasePattern.matchcCs t|dkrdS|j|d|S)z Does this pattern exactly match a sequence of nodes? Default implementation for non-wildcard patterns. rFr)r`rj)rnodesrhr r r match_seqs zBasePattern.match_seqccs&i}|r"|j|d|r"d|fVdS)z} Generator yielding all matches for this pattern. Default implementation for non-wildcard patterns. rrN)rj)rrkrir r rgenerate_matchesszBasePattern.generate_matches)N)N) rCrDrErFrrdr rrQrerjrlrmr r r rrbs  rbc@s*eZdZdddZd ddZd ddZdS) LeafPatternNcCs\|dk r(d|kodkns(t||dk rFt|tsFtt|||_||_||_dS)ap Initializer. Takes optional type, content, and name. The type, if given must be a token type (< 256). If not given, this matches any *leaf* node; the content may still be required. The content, if given, must be a string. If a name is given, the matching node is stored in the results dict under that key. NrrL)rr#r"rMrrdr )rrrdr r r rrPs  zLeafPattern.__init__cCst|tsdStj|||S)z*Override match() to insist on a leaf node.F)r#r.rbrj)rr0rhr r rrj s zLeafPattern.matchcCs |j|jkS)a Match the pattern's content to the node's children. This assumes the node type matches and self.content is not None. Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. When returning False, the results dict may still be updated. )rdr_)rr0rhr r rrfs zLeafPattern._submatch)NNN)N)N)rCrDrErPrjrfr r r rrns  rnc@s$eZdZdZdddZdddZdS) NodePatternFNcCs|dk r|dkst||dk r|t|t s8tt|t|}x:t|D].\}}t|tsht||ft|trJd|_qJW||_ ||_ ||_ dS)ad Initializer. Takes optional type, content, and name. The type, if given, must be a symbol type (>= 256). If the type is None this matches *any* single node (leaf or not), except if content is not None, in which it only matches non-leaf nodes that also match the content pattern. The content, if not None, must be a sequence of Patterns that must match the node's children exactly. If the content is given, the type must not be None. If a name is given, the matching node is stored in the results dict under that key. NrLT) rr#r"rMr$r3rbWildcardPattern wildcardsrrdr )rrrdr r4itemr r rrP%s  zNodePattern.__init__cCs|jrJx>t|j|jD],\}}|t|jkr|dk r>|j|dSqWdSt|jt|jkrbdSx*t|j|jD]\}}|j||srdSqrWdS)a Match the pattern's content to the node's children. This assumes the node type matches and self.content is not None. Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. When returning False, the results dict may still be updated. NTF)rqrmrdr%r`rgziprj)rr0rhcri subpatternr7r r rrfBs   zNodePattern._submatch)NNN)N)rCrDrErqrPrfr r r rro!s roc@s^eZdZdZddedfddZddZddd Zdd d Zd d Z ddZ ddZ ddZ dS)rpa A wildcard pattern can match zero or more nodes. This has all the flexibility needed to implement patterns like: .* .+ .? .{m,n} (a b c | d e | f) (...)* (...)+ (...)? (...){m,n} except it always uses non-greedy matching. NrcCsd|ko|kotkns.t||f|dk rzttt|}t|sXtt|x |D]}t|s^tt|q^W||_||_||_||_ dS)a Initializer. Args: content: optional sequence of subsequences of patterns; if absent, matches one node; if present, each subsequence is an alternative [*] min: optional minimum number of times to match, default 0 max: optional maximum number of times to match, default HUGE name: optional name assigned to this match [*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is equivalent to (a b c | d e | f g h); if content is None, this is equivalent to '.' in regular expression terms. The min and max parameters work as follows: min=0, max=maxint: .* min=1, max=maxint: .+ min=0, max=1: .? min=1, max=1: . If content is not None, replace the dot with the parenthesized list of alternatives, e.g. (a b c | d e | f g h)* rN) HUGErtuplerSr`rMrdminmaxr )rrdrxryr altr r rrPls. zWildcardPattern.__init__cCsd}|jdk r\}}|t|kr |dk rF|j||jrFt|||j<dSq WdS)z4Does this pattern exactly match a sequence of nodes?NTF)rmr`rgr r$)rrkrhrtrir r rrls  zWildcardPattern.match_seqccs:|jdkrXxJt|jdtt||jD]*}i}|jrH|d|||j<||fVq(Wn|jdkrp|j|Vnttdrtj }t t_ zy@x:|j |dD]*\}}|jr|d|||j<||fVqWWnRt k rx:|j |D],\}}|jr |d|||j<||fVqWYnXWdttdr4|t_ XdS)a" Generator yielding matches for a sequence of nodes. Args: nodes: sequence of nodes Yields: (count, results) tuples where: count: the match comprises nodes[:count]; results: dict containing named submatches. NrZ bare_name getrefcountr)rdrangerxr`ryr _bare_name_matcheshasattrrIstderrr_recursive_matches RuntimeError_iterative_matches)rrkcountriZ save_stderrr r rrms. "   z WildcardPattern.generate_matchesc cs t|}d|jkrdifVg}x>|jD]4}x.t||D] \}}||fV|j||fq8Wq(Wx|rg}x|D]\}} ||krr||jkrrxn|jD]d}x^t|||dD]H\} } | dkri}|j| |j| || |fV|j|| |fqWqWqrW|}qbWdS)z(Helper to iteratively yield the matches.rN)r`rxrdrmr'ryrg) rrkZnodelenrhrzrtriZ new_resultsc0r0c1r1r r rrs*       z"WildcardPattern._iterative_matchescCsxd}i}d}t|}xH| r\||kr\d}x0|jD]&}|dj|||r0|d7}d}Pq0WqW|d|||j<||fS)z(Special optimized matcher for bare_name.rFTrN)r`rdrjr )rrkrridoneryZleafr r rr}s  z"WildcardPattern._bare_name_matchesc cs|jdk st||jkr"difV||jkrxr|jD]h}xbt||D]T\}}xJ|j||d|dD].\}}i}|j||j||||fVqfWqDWq4WdS)z(Helper to recursively yield the matches.Nrr)rdrrxryrmrrg) rrkrrzrrrrrir r rr s    "  z"WildcardPattern._recursive_matches)N)N) rCrDrErFrvrPrerjrlrmrr}rr r r rrp^s #  -rpc@s.eZdZd ddZddZddZdd ZdS) NegatedPatternNcCs(|dk rt|tstt|||_dS)a Initializer. The argument is either a pattern or None. If it is None, this only matches an empty sequence (effectively '$' in regex lingo). If it is not None, this matches whenever the argument pattern doesn't have any matches. N)r#rbrrMrd)rrdr r rrPs zNegatedPattern.__init__cCsdS)NFr )rr0r r rrj)szNegatedPattern.matchcCs t|dkS)Nr)r`)rrkr r rrl-szNegatedPattern.match_seqccsL|jdkr"t|dkrHdifVn&x|jj|D] \}}dSWdifVdS)Nr)rdr`rm)rrkrtrir r rrm1s    zNegatedPattern.generate_matches)N)rCrDrErPrjrlrmr r r rrs rc cs|sdifVn|d|dd}}xl|j|D]^\}}|sJ||fVq2xDt|||dD].\}}i}|j||j||||fVq^Wq2WdS)aR Generator yielding matches for a sequence of patterns and nodes. Args: patterns: a sequence of patterns nodes: a sequence of nodes Yields: (count, results) tuples where: count: the entire sequence of patterns matches nodes[:count]; results: dict containing named submatches. rrN)rmrg) Zpatternsrkprestrrrrrir r rrm=s     rm)rF __author__rIwarningsiorrvrrrrrKr.rarbrnrorprrmr r r r s&  1nNV,==#