Testing, Troubleshooting and Deployment
PayPal Instant Payment Notification solutions can be very simple or very complex. An IPN listener could do nothing but send an email notification or it could handle multiple, automated tasks at one time like updating databases and interacting with various third-party web service API’s. Testing and deployment can become a tricky task the more in depth you get with your IPN solution. I’d like to discuss a couple of things immediately that tend to cause problems for developers new to IPN.
Fsock and Curl
As discussed previously, IPN has a verification feature that works by sending the same data back to PayPal so they can verify whether or not it actually came from their servers. If you’re using PHP and you’ve taken a look at the sample code PayPal provides for IPN you’ll notice that they use fsockopen to handle this verification and it looks something like this, although I’ve modified it slightly.
$header = '';
$header .= "POST /cgi-bin/webscr HTTP/1.0\r\n";
$header .= "Host: www.sandbox.paypal.com:443\r\n";
$header .= "Content-Type: application/x-www-form-urlencoded\r\n";
$header .= "Content-Length: " . strlen($req) . "\r\n\r\n";
$fp = fsockopen('ssl://www.sandbox.paypal.com', 443, $errno, $errstr, 30);
fputs($fp, $header . $req);
while(!feof($fp))
{
$res = fgets($fp,1024);
if($strcmp($res, "VERIFIED") == 0)
{
$valid = true;
}
else
{
$valid = false;
}
}
fclose($fp);
Fsockopen works great on most servers, however, some shared servers disable this feature and you will not access to it. In such cases the communication with PayPal will not work and you will get invalid IPN’s every single time. If you run into such a problem you can use curl instead, which I actually went over quickly in Part 1.
// Validate with curl
$req = 'cmd=_notify-validate';
foreach($_POST as $key => $value)
{
$value = urlencode(stripslashes($value));
$req .= '&' . $key . '=' . $value;
}
$curl_result=$curl_err='';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'www.sandbox.paypal.com');
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $req);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded', 'Content-Length: ' . strlen($req)));
curl_setopt($ch, CURLOPT_HEADER , 0);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
$curl_result = @curl_exec($ch);
$curl_err = curl_error($ch);
curl_close($ch);
//Set validation flag
if (strpos($curl_result, 'VERIFIED')!==false)
{
$valid = true;
}
else
{
$valid = false;
}
This will give you exact same result but won’t run into the problems that some shared servers have with fsockopen. Curl is a more widely accepted method and is enabled on any shared server I’ve ever used.
Either way, you’ll notice that a $valid variable is set to true or false based on whether or not the IPN is verified from PayPal’s servers or not. From that point on you can use if($valid) logic to handle tasks appropriately.
IPN Simulator
If you’re building applications for PayPal you’ll need an account at http://developer.paypal.com in order to gain access to the PayPal sandbox using test accounts. You can create multiple buyer and seller accounts within your PayPal developer account and utilize other testing tools provided while building your application prior to launch.
One of the tools provided within your developer account is the IPN Simulator. The IPN Simulator provides an easy way to submit test transactions to your IPN listener script. As we discussed before, IPN is nothing more than POSTed data similar to a form. The IPN Simulator is a simple HTML form which contains the same field names as IPN would send. When submitted you will receive a successful or a failed response.
This can be confusing at times because the IPN simulator will return a successful response message even if the IPN script does not complete successfully. The successful message simply states that the IPN script was successfully hit as opposed to a 404, not found response, for example. So how are we actually supposed to test IPN? I’ll take this time to walk you through the steps I personally like to take when building Instant Payment Notification solutions to avoid headaches along the way.
Testing Techniques
There are various methods to developing and testing IPN. I generally like to follow the same steps each time I begin a new IPN project. It simply keeps me from running into unexpected problems and allows me to maintain a full head of hair during development.
Local Testing – The first thing I like to do after preparing an IPN script is rid it of any basic problems like syntax errors, typos in SQL updates, etc. The easiest way to do this is to see the result of your IPN script happen on screen, however, as we’ve discussed previously, IPN does not take place in a browser, so how can we accomplish this?
Essentially, what I like to do is build my own IPN Simulator. This could be a basic HTML form with actual text fields pre-populated exactly the same way PayPal does it with their simulator, or it could be nothing more than a form full of hidden fields and values. I typically utilize the latter.
Let’s assume I’m building a solution to handle Standard Payments Subscription sign-ups. The following form is filled with hidden fields that resemble the data I can expect to receive from PayPal within IPN.
<form method=”post” action=”http://www.domain.com/paypal-ipn-listener.php”> <input type=”hidden” name=”payment_date” value=”11:02:34 Feb 02, 2009 PST”> <input type=”hidden” name=”txn_type” value=”subscr_signup”> <input type=”hidden” name=”subscr_id” value=”S-5XY9936967688525N”> <input type=”hidden” name=”last_name” value=”Testerson”> <input type=”hidden” name=”residence_country” value=”US”> <input type=”hidden” name=”item_name” value=”Test Membership”> <input type=”hidden” name=”business” value=”sandbo_1215254764_biz@angelleye.com”> <input type=”hidden” name=”amount3″ value=”5.99″> <input type=”hidden” name=”recurring” value=”1″> <input type=”hidden” name=”payer_status” value=”verified”> <input type=”hidden” name=”test_ipn” value=”1″> <input type=”hidden” name=”payer_email” value=”sandbo_1204199080_biz@angelleye.com”> <input type=”hidden” name=”first_name” value=”Drew”> <input type=”hidden” name=”receiver_email” value=”sandbo_1215254764_biz@angelleye.com”> <input type=”hidden” name=”payer_id” value=”E7BTGVXBFSUAU”> <input type=”hidden” name=”reattempt” value=”1″> <input type=”hidden” name=”password” value=”JTB8PgSy6jyiM”> <input type=”hidden” name=”payer_business_name” value=”Drew Angell’s Test Store”> <input type=”hidden” name=”subscr_date” value=”11:02:33 Feb 02, 2009 PST”> <input type=”hidden” name=”username” value=”pp-usurydaze”> <input type=”hidden” name=”period3″ value=”1 M”> <input type=”hidden” name=”mc_amount3″ value=”5.99″> <input type=”submit” name=”Submit” value=”Submit” /> </form>
Notice the action of this form is set to the full URL of the IPN listener I’ve created and uploaded to my web server. I will then upload this form and load it in a web browser. Of course, all we’ll see this point is the submit button because everything else is hidden. Upon pushing the submit button we’ll be able to see the result take place on screen. Any syntax errors, SQL errors, etc. will display and we can get all of those fixed prior to moving on to the next step.
If you’d like to build test forms for different types of IPN’s you can refer to PayPal’s IPN Variable List and see what fields to expect for each.
One thing to note while testing this way is that the IPN will always return INVALID when running through the verification code we discussed before. This, of course, is due to the fact that you’re sending the original data from your own server, not PayPal’s, and their system will be unable to verify it accordingly. You can handle this in the code I provided previously by adjusting it temporarily to the following.
if (strpos($curl_result, 'VERIFIED')!==false)
{
$valid = true;
}
else
{
$valid = true;
}
This way it will pass every time and your code will run through your verified logic instead of the unverified logic.
Once you’re able to submit a local test form and all of your tasks in IPN complete successfully you know you’re ready to move on to the next step in testing.
There really isn’t much to step two in this process. First, re-adjust your IPN code to validate with PayPal correctly again and then return to the IPN Simulator at http://developer.paypal.com. Typically, at this stage you can select an IPN type from the simulator drop-down list and upon submission you see the same back-end tasks occur that occurred when you used a local test form. The only difference is that this time it will actually validate with PayPal’s servers correctly and return VERIFIED as expected.
At this point many people are comfortable, and with good reason, going live with their solution. The simulator acts the same way a real IPN would so if it works there you can generally rest assured that your live transactions will function the same way. I like to take it one step farther and run actual sandbox account tests.
This brings us back to http://developer.paypal.com. This really is a tool for complete testing of PayPal API’s and developer tools, not just IPN, and we can use it to run actual transactions just like we would on the live web site with live PayPal servers.
Let’s stick with the same subscription test we’ve been working with. At your developer account in PayPal make sure you’ve got at least one buyer and one seller test account created. Then, sign into the sandbox seller account which looks and feels just the same as a live paypal.com account. Create a basic subscription button using the wizard exactly the same way you would in my real PayPal account. It returns a simple piece of HTML that you can drop into a web page and allow buyers to sign up. Then, make sure the Instant Payment Notification Preferences in your seller account profile are configured to point to your new IPN solution.
Save the returned HTML into a basic .html file for testing purposes. Upload this page to your web server and then load it in a web browser. At this point you’re able to click on the Subscribe button which will send you over to PayPal (sandbox.paypal.com) and allow you to sign-in and complete the subscription sign-up processing using your sandbox buyer account.
This whole process allows you to see exactly what a buyer would see during checkout and it also “fires” IPN’s exactly the same way the live PayPal servers will. At this point you should see all of the same back-end tasks occur that did during your initial testing but it will happen with the actual transaction data that occurred in the PayPal sandbox test servers.
Once you’re satisfied with the way things are working in the sandbox account you know you’re ready to go live with your IPN solution. There are only a couple of changes you need to make.
First, ensure your Subscription button code (or any button/API code you might be using) is updated to utilize the PayPal live server instead of the sandbox servers. You may have noticed in the code samples provided through-out this article that we’re using www.sandbox.paypal.com instead of the regular www.paypal.com. This needs to be updated when you move live by simply removing sandbox from the host name. Also, if you’re using Standard Hosted Buttons you’ll need to replace the button code you got from your sandbox seller account with the HTML you get from your live PayPal account.
A common mistake people make when going live is forgetting to update the sandbox host values and HTML/API code to the live versions. For example, if you forget to update the IPN verification code the live transactions will be INVALID because they didn’t come from www.sandbox.paypal.com, but from the live www.paypal.com, so be careful to avoid that mistake.
Troubleshooting
Sometimes during development or even after you’ve been live for a while things go wrong and you’re not exactly sure where to start troubleshooting. There are a couple of resources available that can get you pointed in the right direction.
First, PayPal introduced a feature called IPN History about a year ago now. This allows you to see all of the IPN’s that were sent from your PayPal account as well as the response code that PayPal received back from your server. If it’s a 200 OK you know the IPN script at least completed successfully. If it’s anything else you know you’ve got a problem with the script code or the server and can begin to address that.
IPN History can be found by logging into your PayPal account, click into your Profile, select Instant Payment Notification Preferences, and then click the IPN History Page link. Alternatively, if you’re not using Payments Pro, you can simply hover over the History link on your Account Overview page and IPN History will show up in a drop down menu. For some reason these drop-down menus don’t work if you have Payments Pro enabled on your account.
Another good tool for troubleshooting is your web server’s raw log information. Every time your IPN script (or any page on your server for that matter) is requested the result is saved in your server logs. If a syntax error is happening or if the server is timing out you’ll be able to see that in your server logs.
The location of your server logs varies depending on the type of server you’re using. If you have access to CPanel or Plesk you can generally find the RAW Server Logs pretty easily. If you need to track them down on the server directly please refer to Apache and IIS documentation for where to find them.
Conclusion
Instant Payment Notification is a wonderful beast. It can seem confusing and tough to work with but following these guidelines I think you’ll find it much easier to work with and accomplish some powerful tasks.
Just remember: Local Testing, IPN Simulator, Sandbox, Go Live! Good luck and happy developing!