Implementing & Testing SOAP API clients in Ruby

… and check why 5600+ Rails engineers read also this

Implementing & Testing SOAP API clients in Ruby

The bigger your application, the more likely you will need to integrate with less common APIs. This time, we are going to discuss testing communication with SOAP based services. It’s no big deal. Still better than gzipped XMLs over SFTP (I will leave that story to another time).

I am always a bit anxious when I hear SOAP API. It sounds so enterprisey and intimidating. But it doesn’t need to be. Also, I usually prematurely worry that Ruby tooling won’t be good enough to handle all those XMLs. Perhaps this is because of some of my memories of terrible SOAP APIs that I needed to integrate with when I was working as a .NET developer. But SOAP is not inherently evil. In fact, it has some good sides as well.

Implementation

We are going to use savon gem for the implementation and webmock to help us with testing. The plan is to implement a capture functionality for a payment gateway. It means that goods were already shipped or delivered to the customer and the reserved amount can be paid to the merchant.

Let’s see the implementation first and go through it.

def capture(order_id)
  client = Savon.client(
    wsdl:        static_configuration.goods_shipped_url,
    logger:      Rails.logger,
    log_level:   :debug,
    log:         true,
    ssl_version: :TLSv1,
  )

  data = {
    companyID:  static_configuration.company_id.to_s,
    orderID:    order_id,
    retailerID: static_configuration.retailer_id.to_s,
  }.tap do |params|
    params[:signature] = HashGuard.new(
      static_configuration.shared_secret
    ).calculate(params.values)
  end

  response = client.call(
    :goods_shipped,
    message: data,
  )

  result = response.body[:goods_shipped_response][:goods_shipped_result]
  result[:status] == "Ok" or raise CaptureFailed, "Capture status is: #{result[:status]}"
  return result[:TransactionID]
end

The example is not long but sufficient enough to discuss a few aspects.

There is a static configuration that we don’t need to bother ourselves with right now. It contains API URLs and API keys. In Rails app they usually differ per environment. Development and staging are using the pre-production environment of the API provider. Our production env is using API production host. In tests, I usually use pre-production config for safety as well. But thanks to webmock we should never reach this host anyway.

We use Savon gem to communicate with the API. I explicitly configure it to use TLS instead of the obsolete SSL protocol for safety. Depending on your preferences you might set it to log the full communication and to which file. I find it very useful to have full dump during the exploratory phase. When I just play with the API in development to see how it behaves and what it responds. Having full output of the XML from requests and responses can be a lifesaver when debugging and comparing with documentation.

The most important part of the initialization is:

Savon.client(
  wsdl: static_configuration.goods_shipped_url,
)

It tells Savon where to find WSDL - an XML file for describing network services as a set of endpoints operating on messages.

It can be used to descripe messages/types:

This is for example what we need to send:

<s:element name="GoodsShipped">
  <s:complexType>
    <s:sequence>
      <s:element minOccurs="1" maxOccurs="1" name="companyID" type="s:int" />
      <s:element minOccurs="1" maxOccurs="1" name="retailerID" type="s:int" />
      <s:element minOccurs="0" maxOccurs="1" name="orderID" type="s:string" />
      <s:element minOccurs="0" maxOccurs="1" name="signature" type="s:string" />
    </s:sequence>
  </s:complexType>
</s:element>

and this is what we will receive:

<s:complexType name="GoodsShippedResponse">
  <s:sequence>
    <s:element minOccurs="1" maxOccurs="1" name="Status" type="s1:GoodsShippedStatus" />
    <s:element minOccurs="0" maxOccurs="1" name="TransactionID" type="s:string" />
    <s:element minOccurs="1" maxOccurs="1" name="ContractID" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="LoanAmount" type="s:double" />
  </s:sequence>
</s:complexType>

What is a GoodsShippedStatus ?

<s:simpleType name="GoodsShippedStatus">
  <s:restriction base="s:string">
    <s:enumeration value="Ok" />
    <s:enumeration value="WrongState" />
    <s:enumeration value="Error" />
  </s:restriction>
</s:simpleType>

So as you can see the whole API is defined based on primitives which build more complex types which can be parts of even more complex types.

The best thing about using SOAP APIs with WSDL is that the client can parse such API definition and dynamically or statically define all the methods and conversions required to interact with the API.

Also, even when the API documentation written by humans is incorrect, you can peek into the WSDL to see what’s actually going on there. It helped me a lot a few times.

In next part, we build a Hash with keys matching the names from the WSDL definition of the type.

data = {
  companyID:  static_configuration.company_id.to_s,
  orderID:    order_id,
  retailerID: static_configuration.retailer_id.to_s,
}.tap do |params|
  params[:signature] = HashGuard.new(
    static_configuration.shared_secret
  ).calculate(params.values)
end

The signature is a cryptographic digest of all the other values based on a secret that only me and the payment gateway should know. That way the gateway can check the integrity of the message and that it is coming from me and not somebody else. So it plays a role of authentication token as well. I extracted the implementation into HashGuard class which is not interesting for us today.

Finally, we call goods_shipped API endpoint which is also defined in the WSDL so Savon knows how to reach it and how to build the XML with the data that we provide.

response = client.call(
  :goods_shipped,
  message: data,
)

The result of the API call is also automatically converted for us from XML to Ruby primitives such as numbers, strings, arrays and hashes.

result = response.body[:goods_shipped_response][:goods_shipped_result]
result[:status] == "Ok" or raise ::PaymentGateway::Errors::CaptureFailed, "Capture status is: #{result[:status]}"
return result[:TransactionID]

So we can extract the interesting part and see if everything worked correctly.

Testing

I am going to test this code based on the underlying networking communication protocol. In other words, we will stub the HTTP requests with the XML being sent.

This is on purpose. I want to be able to switch to different gem or a library provided by the payment gateway authors without the need to change the tests.

If I just stubbed Ruby method calls, I would not have the ability to change the implementation without changing tests. I would be just typo-testing the implementation. That way I check if we send proper data over the wire and how we react to response data. It does not matter if I use Savon or handcraft those XMLs and URLs myself.

specify "successful capture" do
  stub_getting_wsdl_definition
  stub_request(:post, 'https://example.org/Services/WebshopIntegration.asmx').with(body: <<-XML.split("\n").map(&:strip).join
    <?xml version="1.0" encoding="UTF-8"?>
    <env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tns="http://abc.example.org/" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="http://abc.example.org/OrderStatusdResponse">
      <env:Body>
        <tns:GoodsShipped>
          <tns:companyID>3</tns:companyID>
          <tns:orderID>devz2556219t0r61</tns:orderID>
          <tns:retailerID>9999</tns:retailerID>
          <tns:signature>H1MgHg81vVEVOiJt7ivGz5aVvPM2wIm1GnzTHSqg2m8=</tns:signature>
        </tns:GoodsShipped>
      </env:Body>
    </env:Envelope>
  XML

  ).to_return(:status => 200, :body => body = <<-XML
    <?xml version="1.0" encoding="utf-8"?>
    <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      <soap:Body>
        <GoodsShippedResponse xmlns="http://abc.example.org/">
          <GoodsShippedResult xmlns="http://abc.example.org/GoodsShippedResponse">
            <Status>Ok</Status>
            <TransactionID>8c2ee655b5114</TransactionID>
            <ContractID>1005869</ContractID>
            <LoanAmount>2222</LoanAmount>
          </GoodsShippedResult>
        </GoodsShippedResponse>
      </soap:Body>
    </soap:Envelope>
  XML
  )

  transaction_id = gateway.capture("devz2556219t0r61")
  expect(transaction_id).to eq("8c2ee655b5114")
end

private

def stub_getting_wsdl_definition
  stub_request(:get, "https://i.example.org/Services/WebshopIntegration.asmx?WSDL").
    to_return(
      status: 200, 
      body: Rails.root.join("spec/fixtures/pg.wsdl.xml").read,
    )
end

First, we stub getting the WSDL. I downloaded it myself and saved under spec/fixtures/pg.wsdl.xml. They are usually quite a long files, so I prefer to keep their content outside of the specification. It remains the same and does not depend on any parameters that we could pass so it does not bring anything valuable to the spec.

Then we stub the GoodsShipped request that we issue. It contains the static data coming from the configuration and the provided order_id. I have taken the XML structure of the file from savon logs while playing with the API. Sometimes you have the correct XML structure provided as part of the API documentation.

Notice the body: <<-XML.split("\n").map(&:strip).join part. The XML generated by savon is not pretty formatted. I like my XMLs in tests to be human readable. So I use this little trick to compact my XML into the same format as savon will generate. It has no indentation.

We also stub the response. In this test, we are checking the successful path. So the status is “Ok”. In such case, our adapter should return the transaction_id from the response. That would be 8c2ee655b5114.

Would you like to continue learning more?

If you enjoyed the article, subscribe to our newsletter so that you are always the first one to get the knowledge that you might find useful in your everyday Rails programmer job.

Content is mostly focused on (but not limited to) Ruby, Rails, Web-development and refactoring Rails applications.

You might also like