okay a viral i'm gonna talk about some work often recently so for testing web

services or all the testing the client codes which we use and i still to

web services

so what is web service because of examples to go contacts people you to i

guess a web

open street maps a few also downloading map files facebook scruffy ple basically any hasty

gpus H T S service which

can be and is used by the desktop

i don't specifically limiting my work eight H T P and he cts because it

makes everything a whole is

and doesn't really

restrictive anyway because

pretty much every web service uses one of the same because you don't get problems

of a levels

in that sense


when you think about testing web services you go to the big question you can

ones there is what you want to test

you want to test your point code you wanna make sure isn't crash but you

want tested the point conforms to the current behaviour of the server or some cost

behaviour which is what you actually programmed against

the user would say you want current behaviour because that's what i actually want to

use but that's quite tricky to do because that means you gonna test online which

gives you of those problems with connectivity and race conditions and slowness and all sorts

so we wanna catch regressions in point carried but we also wanna catch that isn't

it changes in the save as behaviour which require changes in your point codes the

new features or

whatever the changed online


you can do it two ways you can do offline testing or online testing and

one talking about here is offline testing developing working on

the bait basic idea of offline testing is that you somehow record how the server

hey and then test against are offline not using that work not touching the curve

javier the server so it's reliable because younger always network problem you've eliminated the web

service and all the network infrastructure in between from your system on the test

so massively cut down a good place

but it's not what users care about because the web service changes your point code

isn't getting tested against like situation

you can also

test against the carburetor the server

which is as i said what we use just care about but it requires a

network connections the instantly eliminates any kind of continuous integration or testing on a seven

provided by know because it's just so unreliable it's hassles that so and network admins

don't like it

it's unreliable

and that's a problem that's it's and me a lot in the G data which

is more based this work on where basically every time a run make check it

failed and that would be D to

transient network problems or subtle changes in the so that it behaviour that S how

to compensated for but the point was actually point

another thing about online testing is that web services change so if you're gonna do

offline testing you need to

be able to change your tests and update them so they represent the lexus takes

of a server and that was easy because you don't wanna spend time doing that

you got much better things today

finally what services are distributed systems which is a problem it's an lots

which means that roger circuit a good wasn't example later just have one server which

has a leo contacts on they have hundreds and as it turns out if you

say for example instead to contact in your address book and it says another one

and then query your address book one you contacts

often they don't here for ten twenty seconds

which means that your test code has to get to sleep the channel twenty seconds

and that's it means your tests take ages to run which is horrible because you

can get most of the way three make check and then it'll five and you

gotta start again

so that's one of the major reasons why online testing bad

there are also other problems with

online testing do you to things like resource limits and

the fact that you have to use not a natural online account so testing against

cable services they limit you to say a thousand press today so if you wanna

run make checking continuous integration you gonna hit against that limits and also new tests

are gonna fail for us today and there's nothing you can do about that

similarly if you wanna run test from multiple locations so one developer in europe and

one america and they both whatever make czech good people is gonna say well that

test account is be involved in two places at once

i'm gonna look at out and then again make czech stops working

so it would be working on is

maybe oscillation for offline testing

where you record H T P messages between client and server you save into a

file you commit that get you then run only a test again start but periodically

you can manually check and compare the trace files against the carburetor was of the

and check other the same or if the difference then you know what's changed so

you cannot take your code in your tests to match but it means that's for

most the time this of one testing can be done in continuous integration it can

be a lot faster sleep and you don't have always problems with synchronization or resource


so there are various other existing solutions i mean it's a pretty simple idea but

most of the large over only so the first three they basically exactly what describes

why more is a separate process which makes in the paying to set up

rest drive a is rest only which eliminates a huge number one services but

also means you can test if you don't know number one services and betamax children

as well

question they will smoke an e-mail of all the open source projects which is similar

sort of things with the cost and hardware but obviously not exceeding gonna translate and


so the approach of taking would look at it is to a broad enough is

a revelation

we directly apply requests to a local in precisely but other and then from that

replay the trace and compare every incoming H T you request with the trains and

then reply with respondent race well

if they don't if they you can make the press that match what's in that

race while then you get never your tests fail and everyone stopping one happy

alternatively you can not a variety S and the test will go straight to the

online server and then you can compare the responses not with the choice while and

see if the servers changed compared to last time you save the choices

so you this operates at H T P level using them so you which is

simple things a lot this transpose can be human readable and

does debug able

it means you can do

simulation of H T P errors like timeouts and for a forcible server iras every

five hundreds things like that with relative ease you not filling around of the lower

levels of T C P I P which would just be too much for fire

it gives you extra power but for the most part it's not needed for more

web services "'cause" there will be H T to you know like

unlike a one or more

the server that's your testing against the offline server is kept in process so you

don't have process synchronisation signal tests and this mock server so some for things a


and because it's overriding the N S it requires minimal code changes to your point

so of the solutions that people

suggest would be to take your clients and split it out into interface isn't and

reimplement interfaces with an online petition offline version and that becomes fourty complicated cost many

changes and is

a real fast

so by writing the N S you basically have to change a client to

specimen H T people to use because it'll be a wide range one so you'd

have to run three but you can easily switch between online and offline just by

turning off the gas reservation

so diagrams everyone much diagrams

so this is have things currently all you've got your system on the test which

is your point code running on the desktop and you've got on one server which

are testing against you got always this page T V transactions going between them


now introduce a lots of the choice while and you locally say C T V

transactions to that race while so we still doing online stuff but once when you

first set of tests you save more to try as well

and commitment to get

you can and the your testing case that race while eliminate on one server everything

goes remote server by redirecting guess the choice well get right into that and of

course happy

you can also compare your choice well against the online so the point again switching

to doing the transactions online but also reading interest well i'm comparing now same time

so you can see what's changed you do not manually once a month or whatever

one of if you like it or something develop to take trough and you continuous

integration so

so he's a example

of how we change your unit test so the test it is here

whatever it is something sensation he request gets response pulses it doesn't ounce bottle of

and what you have is

beginning dedicated and dedicated around each unit test

which was a trace the stock price

it's cool since your code somewhat susceptible to use because you gonna use this high

range pull so you don't wanna use rate don't just use port eighty

and then you also tell the D N S resolver which your is to have

a right so if you testing able services you know growing people dot com and

some supplements that when you set up to run results you'll be back seven

this also means that any requests that you don't expect to other domains however unlikely

that they would gets a D N S translation arrow and so you can catch

buttons that way

probably not gonna happen

and you can wrap as a lot than us and macros or something so you

basically just a one line before and after each unit test

there is some code i'm not showing here which you and you log handler which

will be htk mess in haiti messages like tricks while but that's just a few

lines didn't feel

so i've implemented this village a data which is the project a hard on about


so make czech was over ten minutes because of all these sleeps to wait for

the synchronisation in the street a system on the server side

you can emanate willows and make czech negatives down two hundred and thirty seven seconds

done with a quick run of point make check

you can cover interesting about the same after implementing this it's likely increased because i

did a few tests for every conditions which i can nowadays so i can now

simulates and this of the giving you an access denied error or saying your accounts

been compromised

so you can add test cases for those which you wouldn't be able to do


because you can't manipulate so state

but generally it's been useful

i haven't

applied the store the libraries yes but there's a whole later than can be modified

this that and help also there's the facebook graphic the only as a go a

do you able a no one counts

do the network as it could possibly use it as well this

you know as a lot of stuff that could have it and it was is

become a bit more stable

but there are some problems so

in your traditional unit tests that's a you have a set of function the unit

test function attend on function and you can reuse the set of content and functions

for multiple unit tests because they do common things like you have won the ads

the contact and that at a functional remove contacts and then you have there is

unit tests do different things with a new contract

unfortunately this means that's

if you have a choice while

per unit test

and they have point these in them

not identify your contact for example

the only these are generated by the server will differ from indication of set up

to different invocations that so

and there's a bit of written in the trace files which means that your unit

tests will need

well that response for each unit test believe modifying afterwards to make a list id

is the same

which is something i haven't worked out how to fix yet

probably isn't too hard

similarly you go from the time stamps if you have H T P response which

includes time stamps well in the body or in that is a nice it up

to choice while and then you run test against it next month it's on some

so gonna get acetate and i could cause problems in some cases

again i think is work around

and then also if you may detest against cancellation the messages the current way of

will be noticed that race problems knots

particularly great

and needs a bit of work

so currently this work is being done in the G data and i haven't yet

split up into separate project i'm in the process of doing that and it'll be

committed to get doris shortly

sorry please on other people can start using it

there's a lot of potential running standard error responses way like return an error five

hundred or narrow for a for or whatever and it be good to implements

some kind of easy support for that in they

the testing code so that every point code uses it doesn't have to read that

and goals need to what of the libraries

so that was

representation of it and also construct a bit of a discussion about this because it's

an area when i'm can improve a lot basically most of a networking code websites

is good is

effectively untested that lines so

this is a like a brief implementation of my ideas and i'm sure there's a

lot of stuff can be added an improved

and i'd welcome any suggestions people have

well questions

i do you deal with the be catching a bit encrypted connections

as in H T us encryption


well if i understood well the moat server cash the request

yes that catches the H T P requests so actually yes

packaging of that this is not don't within the tries postal remote server has its

own dummy certificates and K which you have to commit to get and then you

also have to model for your point code to say no don't districts of course

in checks we so there are being work that for that kind of like so

you've done some work in yes i mean would you know to use it H

T yes and also working point obviously you modifications district see a checks would be

the okay like and should never be used in production but

this is a few lines

so i wanted to ask you to will who are offering some kind of staging

services like to just it on production servers or

is there something else

so i don't know about anything else but giggle don't have testing service they have

an online api playground which is web only when you can talk in requests on

a web page and test them out but that's mostly documentation thing so what the

test cases that i have run against productions others and that's what you run into

these problems with and request limits and

accounting not so

and it would be nice if they provided

playground sins a staging service

like that

no questions okay so because will be upon victorious hopefully in the next day at

the moment it's in the G data in the tests directory and wants to take

a look and or controlled we have to is about me

thoughts you have so thank you much

thank you