Gotcha: 2 Preflow Tags

Proxy or Target XML file can contain multiple PreFlow tags. Though XML allows it syntactically - semantically this does not work as expected. The validation is currently missing.

 <PreFlow>
        <Request>
            <Step>
                <Name>assign_client_id</Name>
            </Step>
        </Request>
    </PreFlow>
    <PreFlow>
        <Request>
            <Step>
                <Name>assign_referrer_id</Name>
            </Step>
        </Request>
 </PreFlow>

In case of multiple PreFlow tags, only the policies in the last (textually last) PreFlow tag is executed. The policies in other PreFlow tags are silently ignored. Deployment and runtime validation are missing for this error. In the example above - only assign_referrer_id policy will be executed.

Handling for CLASSIFICATION_FAILURE (404 Handling)

One of the production checklist items while setting up Apigee for production is handling a invalid API call.  The call typically results in CLASSIFICATION_FAILURE error.  This is a stackoverflow question related to issue.

Its always best to have a catch all URL in production environment of Apigee.  You can create brand new proxy with a base path  / . This proxy can have a single flow that returns the custom message using an assign message policy. This catch all proxy is like a custom 404 handler in web-servers.

stdout in Javascript Callout

Writing to console in Javascript callout is useful in some scenarios of debugging.  The print function can be used for writing to stdout.

print('Hello World From Javascript Callout');

would output the message to stdout.

stdout can be seen in trace when you click on the Javascript policy that logged to console.

Flow variables in URL of HTTPTargetConnection

HTTPTargetConnection with URL can be  used in two places as of now - service callout and target endpoint. Following are the configuration differences in both usages.

Service Callout
<HTTPTargetConnection>
    <URL>http://{backend_url}</URL>
</HTTPTargetConnection>

Target Endpoint
<HTTPTargetConnection>
   <URL>http://apigee.com/{backend_path}</URL>
</HTTPTargetConnection>

Details:

• Protocol must be static in both cases it cannot be part of a flow variable used. The validation of requirement is done during deployment time.
• Host (apigee.com in above example) must be static in case of target endpoint. This cannot be part of flow variable; making host part of the flow variable causes failure with error code 500 even though the validation during deployment succeeds.

Handling 404 backend response

In REST API interactions its common to obtain and handle 404 response. The behaviour is used to support create or update scenario for a resources. Check if the resource exists, if it does update it using PUT otherwise create it using POST verb.

404 response from backend is treated as an error in Apigee and such a response results in execution of the fault flow instead of the response flow. The method of dealing with such a scenario is to use the success.codes property of target endpoint properties. Following is an example configuration of target end point to deal with 404 as a success response.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="billing">
    <HTTPTargetConnection>
        <LoadBalancer>
            <Algorithm>RoundRobin</Algorithm>
            <RetryEnabled>true</RetryEnabled>
            <MaxFailures>10</MaxFailures>
            <Server name="BillingTarget">
                <IsEnabled>true</IsEnabled>
            </Server>
        </LoadBalancer>
        <Path>/user/{userid}/bill/{bill_id}</Path>
        <Properties>
            <Property name="success.codes">1XX,2XX,3XX,404</Property>
        </Properties>
        <HealthMonitor>
            <IsEnabled>true</IsEnabled>
            <IntervalInSec>5</IntervalInSec>
            <TCPMonitor>
                <ConnectTimeoutInSec>10</ConnectTimeoutInSec>
                <Port>443</Port>
            </TCPMonitor>
        </HealthMonitor>
    </HTTPTargetConnection>
</TargetEndpoint>

The success codes property above ensures that default behaviour is retained in addition to treating 404 as a success code. This configuration will ensure response flow of the proxy will be executed in case of 404 response too. Appropriate action can be take in the response flow by checking the status code value.

Functionality Provided By Apigee

Following is a simple concise summary (which i like!!) on what Apigee provides by my colleague Michael Bissell on stack overflow.

----
http://stackoverflow.com/a/22901404

Where to put functionality in the flow is usually contextual but there are a few easy things to put in every proxy:

1) Key Management: Using Apigee to manage your API keys and mint your access tokens gives you a couple things; first line of defense for unauthorized apps and automatic analytics about what the developer is doing (getting a high error rate from one app? reach out to them and help them solve their problem proactively).

2) Basic Security Policies: Once you know the App is allowed to access your API there are some simple security policies that should be run on the Apigee layer. Payload enforcement (JSON and XML threat protection, regular expressions to block things like SQL injection or other invasive code). You can also set quotas based on the API Key (different developers getting different levels of access based on the products you associate with their keys). You also want to set spike arrests to keep your API traffic from overwhelming your target server.

3) Response Management: Make sure you strip out unnecessary response headers (cookies, server versions, etc) that aren't relevant to the API Contract. No need to tell your app developers about your target architecture, but it's sometimes hard to suppress those headers from application servers. You may also want rules to block unexpected responses from the target server (500 errors that may contain stack traces for example).

4) Caching: The ability to cache responses in Apigee drives a lot of the rest of "where to do it" questions. But being able to return a Cached response from Apigee can decrease your latency by hundreds of milliseconds improving your transactions per second and your developer/consumer satisfaction. The question now becomes how fine-grained you can get your cached response without having to go to the target server.

Beyond that it becomes "Where is it easiest and most efficient to do a task?" Things like JSON to XML, for example, are easy in Apigee, but they're easy in other platforms that may be running on your backend server, too.

----

Clean-up old revisions of API proxies from Apigee Org

The revisions of proxies tend to grow, specifically in an Apigee org used as development environment. Cleaning old un-deployed revisions not only de-clutters the UI but also eliminates unnecessary storage. At times speeds up deployment process too.

Following is a simple perl script I use to automatically clean up un-deployed versions of the proxy. The perl script use the resty command line client to clean up the revisions it found in the org.

#!/usr/bin/perl
use strict;
use JSON;
use Data::Dumper;

my @proxies=("apiproxy1","apiproxy2", "apiproxy3","apiproxy4");

$\ = "\n";
$user="username";
$passwd="password";

foreach my $p (@proxies) {
 print "Processing proxy: $p";
 my $res=`curl -s https://api.enterprise.apigee.com/v1/o/<orgname>/apis/$p -u $user:$passwd`;

 my $json=decode_json $res;
 my @revs=@{$json->{'revision'}};
 foreach my $rev (@revs) {
  print `curl -s -X DELETE https://api.enterprise.apigee.com/v1/o/<orgname>/apis/$p/revisions/$rev -u $user:$passwd`;
 }
}

JSON payload for a Fault

We need to generate a JSON output in case of fault. RaiseFault policy is appropriate for the same. Following is an example policy
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RaiseFault async="false" continueOnError="false" enabled="true" name="invalid_login">
    <FaultResponse>
        <Set>
            <Payload contentType="application/json" variablePrefix="%" variableSuffix="#">
                {
                    "error": "Incorrect login credentials",
                    "source": "API error"
                }
            </Payload>
            <StatusCode>400</StatusCode>
        </Set>
    </FaultResponse>
</RaiseFault>

Why this so different  - please note the variablePrefix, variableSuffix being present even though the content type is application/json and no variables are used. contentType attribute does not have the same impact as in AssignMessage policy. The presence of {} leads to the interpretation of the data between them as variable name. The variablePrefix and variableSuffix change solves the problem.

OAuthV2 Policy Nuances

Few points to keep in mind while working with OAuthV2 policy.

1.  The XML elements in the OAuthV2 policy take references instead of values. 

In AssignMessage and other policies, variable references are specified using a Ref tag or Ref attribute.

 <QueryParam name="local_grant_type" Ref="grant_type">password</QueryParam>

The content of the QueryParam tag above is a literal value password. Consider the following OAuthV2 policy

 <GrantType>password</GrantType>

 <SupportedGrantTypes>

        <GrantType>password</GrantType>

  </SupportedGrantTypes>

The outer GrantType tag content is a reference to a variable named password and not a literal value. The inner GrantType tag contains password as literal value. All the tags outside SupportedGrantTypes & Attributes tags assume their content as variable references. 

This has been mentioned in the documentation too - however it's easy to miss because of the difference in behaviour with respect to other policies.

2. Password tag in OAuthV2 policy is <PassWord>variable_containing_password</PassWord> - note the capitalization of W in password.


3. Value content inside Attribute tag is either variable name or literal content.

• <Attribute>{phone}</Attribute>  
• <Attribute>phone: {phone}</Attribute>

In the first case the value of the variable phone is stored as the attribute of the access token. However, in the second case no variable interpolation of the phone variable occurs. The attribute will have the literal value phone:{phone} .JavaScript Policy is one of the options to build such a mixed content string.

Target Endpoint Validation Caveats

Validation failures in target endpoint xml files result in error messages that do not detail the root cause. In proxy xml files single quote around conditions is not valid. Consider the following example
<TargetEndpoint name="routetobackend">

  <Flows>
        <Flow name="login">
            <Request>
               <Step>
                    <Name>extract_accept_header</Name>
                </Step>
                 <Step>
                    <Condition>!(local_accept := 'application/json') </Condition>
                    <Name>fault_accept_json_not_found</Name>
                </Step>
            </Request>
            <Response/>
        </Flow>
  </Flows>
  <HTTPTargetConnection>
      <LoadBalancer>
         <Algorithm>RoundRobin</Algorithm>
         <RetryEnabled>true</RetryEnabled>
         <Server name="Some_Target">
            <IsEnabled>true</IsEnabled>
         </Server>
      </LoadBalancer>
    <Path>/some/path</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

The single quote around application/json in the Condition tag is not valid. In this case, the error is obtained during runtime only (not deploy time) even though this is a static validation issue. The runtime error is as follows.

{"fault":{"faultstring":"Unable to send the request to the identified target","detail":{"errorcode":"messaging.runtime.TargetMissing"}}}

As i mentioned earlier the error is not descriptive of the root cause. Hence, in case of routing errors like above, its good to do a recheck on the content of the target endpoint xml before looking into network issues related to the backend.

Generating Password based OAuth Access Token In Response Flow

At times, it's required to generate access token in response flow based some custom validation performed with the backend API. In my scenario, I needed refresh token to be generated too. So I could not use implicit OAuth token generation. The best option in my case was to use password based access token generation.

Consider the following oauth policy to generate password based access token.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<OAuthV2 enabled="true" continueOnError="false" async="false" name="generateAccessToken">
    <Operation>GenerateAccessToken</Operation>
    <AppEndUser>Srikanth</AppEndUser>
    <UserName>sriki</UserName>
    <PassWord>sriki</PassWord>
    <GrantType>grant_type</GrantType>
    <ClientId>sriki_client_id</ClientId>
    <SupportedGrantTypes>
        <GrantType>password</GrantType>
    </SupportedGrantTypes>
</OAuthV2>

Above policy continued to fail with 401 UnAuthorized error even though I had specified correct parameters. The solution for such a scenario (found on discussions with internal experts) is the need for an Authorization request header.

The Authorization header should contain Basic Base64-encoded (clientid:client secret). Following Javascript policy step prior to OAuth access token generation will solve the issue.

var client_id = context.getVariable("local_clientid");
var client_secret = context.getVariable("local_secret");
context.setVariable("request.header.Authorization","Basic "+CryptoJS.enc.Base64.stringify(CryptoJS.enc.Latin1
                                      .parse(client_id + ':' + client_secret)));

JSON Payload generation with variables

Apigee Edge has an AssignMessage policy to generate custom payload. Variable delimiter in policies is curly brace {}. Example: {username}

As expected the curly brace {} is also JSON object delimiter. So generating JSON payload with variables would require change of variable prefix and suffix for Edge variables. The example below shows sample snippet to generate JSON response.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="buildUserDetailsResponse">
    <Set>
        <Payload contentType="text/json" variablePrefix="#" variableSuffix="%">
            {
            "name":"#username%",
            "email":"#email%"
            "adddress":#address%
            }
        </Payload>
    </Set>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <AssignTo createNew="false" transport="http" type="response"/>
</AssignMessage>

Please note the  variablePrefix and  variableSuffix attributes of the payload tag. Further, variablePrefix and  variableSuffix cannot be identical.