ó QŒYc@s˜dZddlZddlmZddlmZddlmZddlmZddlmZde fd „ƒYZ ed e fd „ƒYƒZ dS( s‹Top down operator precedence parser. This is an implementation of Vaughan R. Pratt's "Top Down Operator Precedence" parser. (http://dl.acm.org/citation.cfm?doid=512927.512931). These are some additional resources that help explain the general idea behind a Pratt parser: * http://effbot.org/zone/simple-top-down-parsing.htm * http://javascript.crockford.com/tdop/tdop.html A few notes on the implementation. * All the nud/led tokens are on the Parser class itself, and are dispatched using getattr(). This keeps all the parsing logic contained to a single class. * We use two passes through the data. One to create a list of token, then one pass through the tokens to create the AST. While the lexer actually yields tokens, we convert it to a list so we can easily implement two tokens of lookahead. A previous implementation used a fixed circular buffer, but it was significantly slower. Also, the average jmespath expression typically does not have a large amount of token so this is not an issue. And interestingly enough, creating a token list first is actually faster than consuming from the token iterator one token at a time. iÿÿÿÿN(tlexer(twith_repr_method(tast(t exceptions(tvisitortParsercBsÅeZidd6dd6dd6dd6dd6dd6dd6dd6dd 6dd 6dd 6dd 6d d6dd6dd6dd6dd6dd6dd6dd6dd6dd6dd6dd6d d!6d"d#6d$d%6d&d'6d(d)6Zd*ZiZd+Zdd,„Zd-„Zd.„Zd/„Z dd0„Z d1„Z d2„Z d3„Z d4„Zd5„Zd6„Zd7„Zd8„Zd9„Zd:„Zd;„Zd<„Zd=„Zd>„Zd?„Zd@„ZdA„ZdB„ZdC„ZdD„ZdE„ZdF„Z dG„Z!dH„Z"dI„Z#dJ„Z$dK„Z%dL„Z&dM„Z'dN„Z(dO„Z)dP„Z*dQ„Z+dR„Z,dS„Z-dT„Z.d_dU„Z0dV„Z1dW„Z2dX„Z3dY„Z4dZ„Z5d[„Z6d\„Z7d]„Z8e9d^„ƒZ:RS(`iteoftunquoted_identifiertquoted_identifiertliteraltrbrackettrparentcommatrbracetnumbertcurrenttexpreftcolonitpipeitoritanditeqtgttlttgtetltetnei tflattenitstaritfilteri(tdoti-tnoti2tlbracei7tlbracketi<tlpareni i€cCs/d|_dg||_||_d|_dS(Ni(tNonet tokenizert_tokenst _buffer_sizet_index(tselft lookahead((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt__init__Ns  cCsg|jj|ƒ}|dk r"|S|j|ƒ}||j|s _token_led_%s(RDt_advancetgetattrt_error_nud_tokenRCt BINDING_POWERR#t_error_led_token(R(R;t left_tokent nud_functiontleftt current_tokentledt error_token((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRBvs        cCstj|dƒS(NR=(RR (R(ttoken((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_literal‰scCstj|dƒS(NR=(Rtfield(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_unquoted_identifierŒscCs[tj|dƒ}|jƒdkrW|jdƒ}tjd|d|ddƒ‚n|S(NR=R"iR>s1Quoted identifier not allowed for function names.(RRURCRDRR9(R(RSRURG((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_quoted_identifiers cCsStjƒ}|jƒdkr-tjƒ}n|j|jdƒ}tj||ƒS(NR R(RtidentityRCt_parse_projection_rhsRKtvalue_projection(R(RSROtright((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_staršs  cCs|jtjƒƒS(N(t_token_led_filterRRX(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_filter¢scCs |jƒS(N(t_parse_multi_select_hash(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lbrace¥scCs|jƒ}|jdƒ|S(NR (RBt_match(R(RSR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lparen¨s  cCs;tjtjƒƒ}|j|jdƒ}tj||ƒS(NR(RRRXRYRKt projection(R(RSROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_flatten­scCs#|j|jdƒ}tj|ƒS(NR(RBRKRtnot_expression(R(RStexpr((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_not³scCs©|jƒdkr4|jƒ}|jtjƒ|ƒS|jƒdkr›|jdƒdkr›|jƒ|jƒ|j|jdƒ}tj tjƒ|ƒS|j ƒSdS(NRRRiR (snumberR( RCt_parse_index_expressiont_project_if_sliceRRXt _lookaheadRHRYRKRct_parse_multi_select_list(R(RSR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lbracket·s   cCso|jdƒdks*|jdƒdkr4|jƒStj|jdƒdƒ}|jƒ|jdƒ|SdS(NiRiR=R (Rjt_parse_slice_expressionRtindexRDRHRa(R(tnode((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRhÈs   cCsûdddg}d}|jƒ}x½|dk rà|dkrà|dkrˆ|d7}|dkr{|j|jdƒdƒn|jƒnI|dkr¸|jdƒd||<|jƒn|j|jdƒdƒ|jƒ}q$W|jdƒtj|ŒS( NiR iRis syntax errorRR=(R#RCt_raise_parse_error_for_tokenRDRHRaRtslice(R(tpartsRnRP((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRm×s$        cCs tjƒS(N(Rt current_node(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_currentïscCs#|j|jdƒ}tj|ƒS(NR(RBRKRR(R(RSR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_exprefòscCs—|jƒdksc|j|jdƒ}|ddkrM|dj|ƒ|Stj||gƒSn0|jƒ|j|jdƒ}tj||ƒSdS(NRRR>t subexpressiontchildren( RCt_parse_dot_rhsRKtappendRRvRHRYRZ(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_dotös cCs&|j|jdƒ}tj||ƒS(NR(RBRKRR(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_pipescCs&|j|jdƒ}tj||ƒS(NR(RBRKRt or_expression(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_or scCs&|j|jdƒ}tj||ƒS(NR(RBRKRtand_expression(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_and scCsÕ|ddkrN|jdƒ}tj|d|d|dd|dƒ‚n|d}g}xQ|jƒdks±|jƒ}|jƒdkr¡|jdƒn|j|ƒqaW|jdƒtj||ƒ}|S( NR>RUiþÿÿÿR<R=sInvalid function name '%s'R R ( RDRR9RCRBRaRyRtfunction_expression(R(ROtprev_ttnametargsR1t function_node((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lparens   cCsf|jdƒ}|jdƒ|jƒdkr=tjƒ}n|j|jdƒ}tj|||ƒS(NiR RR(RBRaRCRRXRYRKtfilter_projection(R(ROt conditionR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR]%s  cCs|j|dƒS(NR(t_parse_comparator(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_eq/scCs|j|dƒS(NR(Rˆ(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_ne2scCs|j|dƒS(NR(Rˆ(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_gt5scCs|j|dƒS(NR(Rˆ(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_gte8scCs|j|dƒS(NR(Rˆ(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_lt;scCs|j|dƒS(NR(Rˆ(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lte>scCs5tj|ƒ}|j|jdƒ}tj||ƒS(NR(RRRYRKRc(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_flattenAscCs§|jdƒ}|dd krc|jƒ}|ddkrP|dj|ƒ|S|j||ƒSn@|jdƒ|jdƒ|j|jdƒ}tj||ƒSdS( NiR>RRtindex_expressionRwRR (snumberscolon( RDRhRyRiRaRYRKRRc(R(RORSR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lbracketGs   cCsMtj||gƒ}|ddkrEtj||j|jdƒƒS|SdS(NR>RqR(RRRcRYRK(R(ROR[t index_expr((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRiZs cCs)|j|j|ƒ}tj|||ƒS(N(RBRKRt comparator(R(ROR“R[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRˆcscCsig}xFtrN|jƒ}|j|ƒ|jƒdkr>Pq |jdƒq W|jdƒtj|ƒS(NR R (tTrueRBRyRCRaRtmulti_select_list(R(t expressionsR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRkgs    cCsØg}x¿trÇ|jdƒ}|jdddgƒ|d}|jdƒ|jdƒ}tjd|d|ƒ}|j|ƒ|jƒd kr¡|jd ƒq |jƒd kr |jd ƒPq q Wtj d |ƒS( Nit token_typesRRR=Rtkey_nameRoR R tnodes( R”RDt_match_multiple_tokensRaRBRt key_val_pairRyRCtmulti_select_dict(R(tpairst key_tokenR˜R=Ro((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR_ss        cCsÁ|j|jƒ|jkr+tjƒ}n’|jƒdkrO|j|ƒ}nn|jƒdkrs|j|ƒ}nJ|jƒdkr¤|jdƒ|j|ƒ}n|j|j dƒdƒ|S(NR!RRis syntax error( RKRCt_PROJECTION_STOPRRXRBRaRxRpRD(R(R;R[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRY‡s cCs´|jƒ}|d kr%|j|ƒS|dkrH|jdƒ|jƒS|dkrk|jdƒ|jƒS|jdƒ}ddddg}d||df}|j||ƒdS( NRRRR!R isExpecting: %s, got: %sR>(squoted_identifiersunquoted_identifiersstar(RCRBRaRkR_RDRp(R(R;R)RGtallowedtmsg((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRx˜s          cCsH|ddkr4tj|d|d|dƒ‚n|j|dƒdS(NR>RR<R=s invalid token(RR7Rp(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRJ´scCs|j|dƒdS(Ns invalid token(Rp(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRLºscCs<|jƒ|kr|jƒn|j||jdƒƒdS(Ni(RCRHt_raise_parse_error_maybe_eofRD(R(t token_type((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRa½s cCs<|jƒ|kr.|j||jdƒƒn|jƒdS(Ni(RCR¢RDRH(R(R—((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRšÆscCs|jd7_dS(Ni(R'(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRHÌscCs|j|jdS(NR>(R%R'(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRCÏscCs|j|j|dS(NR>(R%R'(R(R((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRjÒscCs|j|j|S(N(R%R'(R(R((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRDÕscCs:|d}|d}|d}tj||||ƒ‚dS(NR<R=R>(RR9(R(RStreasont lex_positiont actual_valuet actual_type((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRpØs     cCsn|d}|d}|d}|dkrBtj|||ƒ‚nd||f}tj||||ƒ‚dS(NR<R=R>RsExpecting: %s, got: %s(RR7R9(R(t expected_typeRSR¥R¦R§tmessage((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR¢ßs     cCsAx:tj|jjƒt|jdƒƒD]}|j|=q)WdS(Ni(trandomtsampleR+tkeystintR/(R(tkey((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR0ës/cCs|jjƒdS(s'Clear the expression compilation cache.N(R+tclear(tcls((s3/usr/lib/python2.7/site-packages/jmespath/parser.pytpurgeïsN(;t__name__t __module__RKRŸR+R/R*R4R-R5RBRTRVRWR\R^R`RbRdRgRlRhRmRtRuRzR{R}RR…R]R‰RŠR‹RŒRRŽRR‘RiRˆRkR_RYRxRJRLR#RaRšRHRCRjRDRpR¢R0t classmethodR±(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR%s¨                                          REcBs/eZd„Zdd„Zd„Zd„ZRS(cCs||_||_dS(N(R1RF(R(R1RF((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR*÷s cCs(tj|ƒ}|j|j|ƒ}|S(N(RtTreeInterpretertvisitRF(R(R=toptionst interpretertresult((s3/usr/lib/python2.7/site-packages/jmespath/parser.pytsearchûscCs"tjƒ}|j|jƒ}|S(sfRender the parsed AST as a dot file. Note that this is marked as an internal method because the AST is an implementation detail and is subject to change. This method can be used to help troubleshoot or for development purposes, but is not considered part of the public supported API. Use at your own risk. (RtGraphvizVisitorR¶RF(R(trenderertcontents((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_render_dot_files cCs t|jƒS(N(treprRF(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt__repr__sN(R²R³R*R#RºR¾RÀ(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyREõs   ( t__doc__RªtjmespathRtjmespath.compatRRRRtobjectRRE(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyts ÿÑ