0:00:09okay a viral i'm gonna talk about some work often recently so for testing web
0:00:16services or all the testing the client codes which we use and i still to
0:00:20web services
0:00:23so what is web service because of examples to go contacts people you to i
0:00:29guess a web
0:00:30open street maps a few also downloading map files facebook scruffy ple basically any hasty
0:00:37gpus H T S service which
0:00:40can be and is used by the desktop
0:00:43i don't specifically limiting my work eight H T P and he cts because it
0:00:47makes everything a whole is
0:00:50and doesn't really
0:00:52restrictive anyway because
0:00:54pretty much every web service uses one of the same because you don't get problems
0:00:58of a levels
0:00:59in that sense
0:01:02so
0:01:04when you think about testing web services you go to the big question you can
0:01:07ones there is what you want to test
0:01:10you want to test your point code you wanna make sure isn't crash but you
0:01:14want tested the point conforms to the current behaviour of the server or some cost
0:01:19behaviour which is what you actually programmed against
0:01:22the user would say you want current behaviour because that's what i actually want to
0:01:25use but that's quite tricky to do because that means you gonna test online which
0:01:31gives you of those problems with connectivity and race conditions and slowness and all sorts
0:01:39so we wanna catch regressions in point carried but we also wanna catch that isn't
0:01:44it changes in the save as behaviour which require changes in your point codes the
0:01:49new features or
0:01:51whatever the changed online
0:01:56so
0:01:58you can do it two ways you can do offline testing or online testing and
0:02:02one talking about here is offline testing developing working on
0:02:08the bait basic idea of offline testing is that you somehow record how the server
0:02:12hey and then test against are offline not using that work not touching the curve
0:02:18javier the server so it's reliable because younger always network problem you've eliminated the web
0:02:24service and all the network infrastructure in between from your system on the test
0:02:29so massively cut down a good place
0:02:32but it's not what users care about because the web service changes your point code
0:02:36isn't getting tested against like situation
0:02:40you can also
0:02:42test against the carburetor the server
0:02:45which is as i said what we use just care about but it requires a
0:02:47network connections the instantly eliminates any kind of continuous integration or testing on a seven
0:02:53provided by know because it's just so unreliable it's hassles that so and network admins
0:02:59don't like it
0:03:00it's unreliable
0:03:02and that's a problem that's it's and me a lot in the G data which
0:03:06is more based this work on where basically every time a run make check it
0:03:11failed and that would be D to
0:03:13transient network problems or subtle changes in the so that it behaviour that S how
0:03:19to compensated for but the point was actually point
0:03:25another thing about online testing is that web services change so if you're gonna do
0:03:30offline testing you need to
0:03:32be able to change your tests and update them so they represent the lexus takes
0:03:37of a server and that was easy because you don't wanna spend time doing that
0:03:40you got much better things today
0:03:44finally what services are distributed systems which is a problem it's an lots
0:03:50which means that roger circuit a good wasn't example later just have one server which
0:03:55has a leo contacts on they have hundreds and as it turns out if you
0:03:59say for example instead to contact in your address book and it says another one
0:04:04and then query your address book one you contacts
0:04:07often they don't here for ten twenty seconds
0:04:12which means that your test code has to get to sleep the channel twenty seconds
0:04:15and that's it means your tests take ages to run which is horrible because you
0:04:20can get most of the way three make check and then it'll five and you
0:04:24gotta start again
0:04:26so that's one of the major reasons why online testing bad
0:04:33there are also other problems with
0:04:37online testing do you to things like resource limits and
0:04:42the fact that you have to use not a natural online account so testing against
0:04:46cable services they limit you to say a thousand press today so if you wanna
0:04:50run make checking continuous integration you gonna hit against that limits and also new tests
0:04:55are gonna fail for us today and there's nothing you can do about that
0:04:58similarly if you wanna run test from multiple locations so one developer in europe and
0:05:03one america and they both whatever make czech good people is gonna say well that
0:05:09test account is be involved in two places at once
0:05:12i'm gonna look at out and then again make czech stops working
0:05:17so it would be working on is
0:05:20maybe oscillation for offline testing
0:05:25where you record H T P messages between client and server you save into a
0:05:29file you commit that get you then run only a test again start but periodically
0:05:34you can manually check and compare the trace files against the carburetor was of the
0:05:40and check other the same or if the difference then you know what's changed so
0:05:44you cannot take your code in your tests to match but it means that's for
0:05:48most the time this of one testing can be done in continuous integration it can
0:05:52be a lot faster sleep and you don't have always problems with synchronization or resource
0:06:00limits
0:06:04so there are various other existing solutions i mean it's a pretty simple idea but
0:06:09most of the large over only so the first three they basically exactly what describes
0:06:16why more is a separate process which makes in the paying to set up
0:06:21rest drive a is rest only which eliminates a huge number one services but
0:06:25also means you can test if you don't know number one services and betamax children
0:06:30as well
0:06:31question they will smoke an e-mail of all the open source projects which is similar
0:06:36sort of things with the cost and hardware but obviously not exceeding gonna translate and
0:06:40fractions
0:06:43so the approach of taking would look at it is to a broad enough is
0:06:48a revelation
0:06:50we directly apply requests to a local in precisely but other and then from that
0:06:56replay the trace and compare every incoming H T you request with the trains and
0:07:01then reply with respondent race well
0:07:04if they don't if they you can make the press that match what's in that
0:07:07race while then you get never your tests fail and everyone stopping one happy
0:07:14alternatively you can not a variety S and the test will go straight to the
0:07:18online server and then you can compare the responses not with the choice while and
0:07:23see if the servers changed compared to last time you save the choices
0:07:29so you this operates at H T P level using them so you which is
0:07:34simple things a lot this transpose can be human readable and
0:07:37does debug able
0:07:40it means you can do
0:07:44simulation of H T P errors like timeouts and for a forcible server iras every
0:07:49five hundreds things like that with relative ease you not filling around of the lower
0:07:54levels of T C P I P which would just be too much for fire
0:07:56it gives you extra power but for the most part it's not needed for more
0:08:00web services "'cause" there will be H T to you know like
0:08:04unlike a one or more
0:08:07the server that's your testing against the offline server is kept in process so you
0:08:13don't have process synchronisation signal tests and this mock server so some for things a
0:08:19lot
0:08:23and because it's overriding the N S it requires minimal code changes to your point
0:08:28so of the solutions that people
0:08:32suggest would be to take your clients and split it out into interface isn't and
0:08:36reimplement interfaces with an online petition offline version and that becomes fourty complicated cost many
0:08:42changes and is
0:08:44a real fast
0:08:47so by writing the N S you basically have to change a client to
0:08:52specimen H T people to use because it'll be a wide range one so you'd
0:08:56have to run three but you can easily switch between online and offline just by
0:09:01turning off the gas reservation
0:09:04so diagrams everyone much diagrams
0:09:07so this is have things currently all you've got your system on the test which
0:09:11is your point code running on the desktop and you've got on one server which
0:09:14are testing against you got always this page T V transactions going between them
0:09:20so
0:09:21now introduce a lots of the choice while and you locally say C T V
0:09:26transactions to that race while so we still doing online stuff but once when you
0:09:31first set of tests you save more to try as well
0:09:35and commitment to get
0:09:38you can and the your testing case that race while eliminate on one server everything
0:09:42goes remote server by redirecting guess the choice well get right into that and of
0:09:48course happy
0:09:50you can also compare your choice well against the online so the point again switching
0:09:55to doing the transactions online but also reading interest well i'm comparing now same time
0:10:01so you can see what's changed you do not manually once a month or whatever
0:10:05one of if you like it or something develop to take trough and you continuous
0:10:09integration so
0:10:12so he's a example
0:10:15of how we change your unit test so the test it is here
0:10:20whatever it is something sensation he request gets response pulses it doesn't ounce bottle of
0:10:26and what you have is
0:10:29beginning dedicated and dedicated around each unit test
0:10:33which was a trace the stock price
0:10:36it's cool since your code somewhat susceptible to use because you gonna use this high
0:10:42range pull so you don't wanna use rate don't just use port eighty
0:10:47and then you also tell the D N S resolver which your is to have
0:10:53a right so if you testing able services you know growing people dot com and
0:10:57some supplements that when you set up to run results you'll be back seven
0:11:02this also means that any requests that you don't expect to other domains however unlikely
0:11:08that they would gets a D N S translation arrow and so you can catch
0:11:12buttons that way
0:11:15probably not gonna happen
0:11:17and you can wrap as a lot than us and macros or something so you
0:11:21basically just a one line before and after each unit test
0:11:25there is some code i'm not showing here which you and you log handler which
0:11:30will be htk mess in haiti messages like tricks while but that's just a few
0:11:36lines didn't feel
0:11:40so i've implemented this village a data which is the project a hard on about
0:11:45most
0:11:47so make czech was over ten minutes because of all these sleeps to wait for
0:11:51the synchronisation in the street a system on the server side
0:11:55you can emanate willows and make czech negatives down two hundred and thirty seven seconds
0:12:00done with a quick run of point make check
0:12:03you can cover interesting about the same after implementing this it's likely increased because i
0:12:08did a few tests for every conditions which i can nowadays so i can now
0:12:12simulates and this of the giving you an access denied error or saying your accounts
0:12:17been compromised
0:12:20so you can add test cases for those which you wouldn't be able to do
0:12:22for
0:12:24because you can't manipulate so state
0:12:27but generally it's been useful
0:12:31i haven't
0:12:32applied the store the libraries yes but there's a whole later than can be modified
0:12:36this that and help also there's the facebook graphic the only as a go a
0:12:42do you able a no one counts
0:12:46do the network as it could possibly use it as well this
0:12:49you know as a lot of stuff that could have it and it was is
0:12:52become a bit more stable
0:12:55but there are some problems so
0:12:59in your traditional unit tests that's a you have a set of function the unit
0:13:03test function attend on function and you can reuse the set of content and functions
0:13:07for multiple unit tests because they do common things like you have won the ads
0:13:12the contact and that at a functional remove contacts and then you have there is
0:13:16unit tests do different things with a new contract
0:13:21unfortunately this means that's
0:13:23if you have a choice while
0:13:26per unit test
0:13:28and they have point these in them
0:13:30not identify your contact for example
0:13:34the only these are generated by the server will differ from indication of set up
0:13:38to different invocations that so
0:13:40and there's a bit of written in the trace files which means that your unit
0:13:44tests will need
0:13:46well that response for each unit test believe modifying afterwards to make a list id
0:13:51is the same
0:13:52which is something i haven't worked out how to fix yet
0:13:56probably isn't too hard
0:13:59similarly you go from the time stamps if you have H T P response which
0:14:03includes time stamps well in the body or in that is a nice it up
0:14:07to choice while and then you run test against it next month it's on some
0:14:11so gonna get acetate and i could cause problems in some cases
0:14:16again i think is work around
0:14:19and then also if you may detest against cancellation the messages the current way of
0:14:23will be noticed that race problems knots
0:14:26particularly great
0:14:29and needs a bit of work
0:14:33so currently this work is being done in the G data and i haven't yet
0:14:37split up into separate project i'm in the process of doing that and it'll be
0:14:41committed to get doris shortly
0:14:44sorry please on other people can start using it
0:14:49there's a lot of potential running standard error responses way like return an error five
0:14:54hundred or narrow for a for or whatever and it be good to implements
0:15:00some kind of easy support for that in they
0:15:02the testing code so that every point code uses it doesn't have to read that
0:15:07and goals need to what of the libraries
0:15:11so that was
0:15:14representation of it and also construct a bit of a discussion about this because it's
0:15:18an area when i'm can improve a lot basically most of a networking code websites
0:15:22is good is
0:15:24effectively untested that lines so
0:15:27this is a like a brief implementation of my ideas and i'm sure there's a
0:15:32lot of stuff can be added an improved
0:15:34and i'd welcome any suggestions people have
0:15:37well questions
0:15:51i do you deal with the be catching a bit encrypted connections
0:15:58as in H T us encryption
0:16:01catching
0:16:04well if i understood well the moat server cash the request
0:16:09yes that catches the H T P requests so actually yes
0:16:16packaging of that this is not don't within the tries postal remote server has its
0:16:22own dummy certificates and K which you have to commit to get and then you
0:16:28also have to model for your point code to say no don't districts of course
0:16:33in checks we so there are being work that for that kind of like so
0:16:39you've done some work in yes i mean would you know to use it H
0:16:43T yes and also working point obviously you modifications district see a checks would be
0:16:49the okay like and should never be used in production but
0:16:52this is a few lines
0:17:07so i wanted to ask you to will who are offering some kind of staging
0:17:12services like to just it on production servers or
0:17:18is there something else
0:17:21so i don't know about anything else but giggle don't have testing service they have
0:17:26an online api playground which is web only when you can talk in requests on
0:17:31a web page and test them out but that's mostly documentation thing so what the
0:17:37test cases that i have run against productions others and that's what you run into
0:17:40these problems with and request limits and
0:17:43accounting not so
0:17:45and it would be nice if they provided
0:17:48playground sins a staging service
0:17:51like that
0:18:01no questions okay so because will be upon victorious hopefully in the next day at
0:18:08the moment it's in the G data in the tests directory and wants to take
0:18:13a look and or controlled we have to is about me
0:18:17thoughts you have so thank you much
0:18:19thank you