Akvo PayPal Under the Hood
From Akvo Labs
Contents |
Introduction
Several enhancements have been integrated into the RSR PayPal engine since its initial release. This document aims to provide a detailed description of what is going on under the hood.
Settings
Several new settings have been added in RSR 0.9.11 which control the behaviour of the PayPal engine.
Some settings have also been deprecated as their functionality has been taken over by new controls in the RSR admin interface.
New settings
Required
DECIMALS_DEBUG = True
This setting controls the display of decimal numbers in the website front end. It has no effect on financial data in the database itself, only on the display of this data. More information is given in the Decimalisation section below.
This setting is referred to in various places in the RSR codebase and is, therefore, required.
PAYPAL_SANDBOX_GATEWAY = 'paypal_sandbox_account@wherever.com'
This setting is required for testing the RSR PayPal engine with the official PayPal Developer Sandbox. It replaces the previous PAYPAL_SANDBOX_BUSINESS setting.
PAYPAL_RECEIVER_EMAIL = ''
This setting is required by the django-paypal library used by the RSR PayPal engine. However, since it is set dynamically by RSR based on a project's PayPal Gateway, it should be defined as an empty string in settings.py. It is likely that this setting will be fully deprecated in future releases.
In testing, the functionality of this setting is provided by PAYPAL_SANDBOX_GATEWAY above. The latter is, again, a required setting.
Optional
DECIMALS_DECIMAL_PLACES = 2
Internally, RSR uses Python Decimal objects to store and represent funding amounts. Since Python Decimal objects only display values with trailing decimal places if the value cannot be represented as a whole number, we must make sure we render all financial data uniformly in the front end - eg Decimal(100) should be rendered as '€100.00' and Decimal('199.99') should be rendered as '€199.99'. This is an internationally accepted convention when displaying financial data.
The display of funding information is now handled by a custom template filter, detailed below. The purpose of this filter is to make sure all funding amounts are displayed in the same way - that is, always displayed with two decimal places if DECIMALS_DEBUG is True. Without this filter, Decimal(100) would be displayed as €100 not as €100.00.
It is unlikely that anyone would want to use a value other than 2 for this setting. However, for the sake of completeness, changing the 2 to a 3 would mean that the previous examples would be displayed as €100.000 and €199.990 respectively.
Deprecated settings
PAYPAL_CURRENCY_CODE = 'EUR' PAYPAL_BUSINESS = 'paypal_account@wherever.com" PAYPAL_SANDBOX_BUSINESS = 'paypal_sandbox_account@wherever.com'
The functionality of PAYPAL_CURRENCY_CODE and PAYPAL_BUSINESS is now covered by the new Gateways feature, described in more detail in its own section below.
PAYPAL_SANDBOX_BUSINESS has been renamed to PAYPAL_SANDBOX_GATEWAY and is described in Required settings above.
PAYPAL_SANDBOX_NOTIFY_URL PAYPAL_SANDBOX_RETURN_URL PAYPAL_SANDBOX_CANCEL_URL
These have been deprecated since their functionality is also provided by PAYPAL_NOTIFY_URL, PAYPAL_RETURN_URL and PAYPAL_CANCEL_URL respectively.
PAYPAL_SANDBOX_PRODUCT_DESCRIPTION_PREFIX
This setting is now derived from PAYPAL_PRODUCT_DESCRIPTION_PREFIX in the donate view function and is now redundant.
Decimalisation
Overview
Akvo has, up until now, dealt internally with whole numbers (integers). As from 0.9.11 (Jupiter) it is entirely decimalised. This was a necessary transition in that payment handlers such as PayPal deduct charges from all donations made which result in a donation of €1 finally being received as €0.61, as a quick example. Therefore, RSR has to handle financial data down to the very last cent. This is, of course, prudent in the long run, in any case.
As a result of this transition, funding calculations are now accurate to the cent whereas previously they could only be represented as whole numbers.
Implementation
Although RSR always deals internally with decimal numbers, the website front end can be configured to display these or not. A new setting controls the presentation of decimals in the website front end via a custom template filter in RSR:
DECIMALS_DEBUG = True
If this setting is True, then financial data will be displayed in its decimal form on the website. If set to False, decimal numbers will be rounded to the nearest whole number and displayed without any decimal places. Some examples follow:
999.49 becomes 999
999.50 becomes 1000
NB: A user may still only donate whole number amounts to projects.
The rationale behind this ability to control display of decimals is that whole numbers are "friendlier" to end users and also take up less space. During testing, however, it is desirable to have access to exact financial data.
Rounding of decimal numbers has several technical and usability implications described in the next section.
Technical implications
There are two technical issues which conspire in tandem:
- Although RSR itself is now decimalised and able to handle amounts of money in cents, users are only allowed to donate whole Euro/Dollar amounts to projects. That is, it is impossible to donate €2.50, only €2 or €3. Also, we have opted to display only whole numbers in the website front end.
- RSR currently cannot predict how much PayPal will deduct in fees from a donation.
In short, although internally RSR works with decimals, we never expose them to the user.
The outcome of this is that It is currently impossible to fully fund a project via individual donations. The direct implication of this is that the maximum amount a user may donate to a project is currently the project's total budget minus 1.
In order to ensure that the funding totals displayed in the website front end add up and do not mislead or confuse the end user, a couple of calculations had to be "massaged" slightly to compensate for this problem. In a nutshell: if a project requires less than 1 whole Euro to meet its funding targets, the front end will declare the project to be fully funded and the user will then not be able to donate anything to it. The project's total raised will then be rounded up so that the Raised and Needed amounts equal the Total Budget, simply to prevent confusion.
It is important to note that the calculations used in the front end do not in any way alter the data in the database. The funding information stored in the database will always accurately reflect a project's actual funding requirements.
Usability implications
Rounding
When decimal numbers are rounded up (as opposed to rounded down) in the website front end, the funding information for the project can often be slightly misleading. For example, if a project requires €2.93 to meet its funding target, the front end will display this number as €3. If the user then attempts to donate €3, the donation form will not validate because €3 > €2.93. The maximum whole number amount a user may actually donate to the project would, in fact, be €2. This has potential for significant confusion.
This situation is compounded by the fact that the error message displayed to the user will be "You cannot donate more than the project needs".
PayPal charges
In future releases of RSR we will include code which can predict the amount of money PayPal will charge for each donation and provide an option on the donation form to fully fund a project. Right now, however, no matter how much money a user donates, PayPal will deduct a percentage fee thus continually eating away at the actual amount donated. In effect, it is essentially impossible, right now, to fully fund a project via individual donations.
As a workaround, RSR currently will not allow the user to donate more than the amount the project still requires minus 1. The funding totals in the front end are rationalised to compensate for this problem.
If we take the previous example, where a project required €2.93 to meet its funding targets, if a user donates €2 the project will immediately then declare that it is fully funded, since it is impossible to donate the final 93 Euro cents.
Both of these usability quirks will be addressed in an upcoming release of RSR.
Gateways
Overview
New in RSR 0.9.11 is the ability to define a PayPal account (or Gateway) to send donations to on a project-by-project basis.
The currency and locale to be used during the donation process are also defined within the Gateway.
Newly created projects default to using the Akvo PayPal account with no further configuration required.
Default gateway
Currently all projects in RSR use the default Akvo PayPal gateway.
This gateway is created during database migration and all projects are updated to use it by default. No further action is required.
Adding a new gateway
Directing donations to an alternative PayPal gateway is a two-stage process and may only be performed by an RSR administrator. When a new project is created in the RSR admin, it will automatically and transparently be assigned to the default Akvo PayPal store.
To override the default store:
- Create a new gateway
- Edit the gateway configuration for the project
To create a new gateway, log into the RSR admin as an administrator and go to "PayPal gateways" and click Add. Make sure you fill in all the required fields then click Save. Help text is provided.
To assign the newly created gateway to a project, go to "Project Paypal gateway configuration" and select the project you want to assign the new gateway to. Choose the gateway you created in Step 1 from the drop down list and then click Save. Donations to this project will now be directed to the alternative PayPal store and donation notification emails will be sent to the email address nominated there.
Template filter
All financial data must now be passed through the custom "round" template filter which is new in 0.9.11. The purpose of this filter is to ensure that all financial data pulled from the database is displayed in a uniform way.
Where previously you would have written something like this:
{{ p.budget_total }}
You now need to write this:
{% load rsr_filters %}
...
{{ p.budget_total|round }}
The behaviour of the "round" filter is controlled by the settings DECIMALS_DEBUG and DECIMALS_DECIMAL_PLACES which are described above.
Technical notes
When using SQLite in development, caution is required when performing sum operations on decimal values since SQLite does not handle this natively. SQLite either produces integers or floats depending on whether or not the result can be expressed as a whole number. To compensate for this, it is sometimes necessary to rationalise sum values into decimals first. You will notice constructions like the following in several places in RSR:
value = Decimal(str(value))
An integer can be translated into a decimal simply:
i = 100 d = Decimal(i)
or:
s = '100' d = Decimal(s)
However, a float must be passed in as a string:
f = 21.6899999 d = Decimal(str(f))
Since an integer value can be passed in either as it stands or as a string and a decimal can also be converted back to a decimal from its string representation, the construction Decimal(str(value)) catches all possible input cases and yields a uniform decimal result regardless of which SQL database system is in use and is less expensive than checking type conditionally.
Specifically, this is to compensate for situations where only SQLite produces a float as the result of summing two or more decimals. Neither MySQL or PostgreSQL are affected by this limitation. It is included because the RSR codebase must also be able to run locally on developers' workstations.
