archive-info.com » INFO » H » HSKUPIN.INFO

Total: 368

Choose link from "Titles, links and description words view":

Or switch to "Titles and links view".
  • firefox | hskupin.info
    will this look like and how will mozmill ci cope with that For the latter I can say that we don t want to run more tests as we do right now This is mostly due to our limited infrastructure I have to maintain myself Having the needs to run firefox ui tests for each check in on all platforms and even for try pushes would mean that we totally exceed the machine capacity Therefore we continue to use mozmill ci for now to test nightly and release builds for en US but also a couple of other locales This might change later this year when mozmill ci can be replaced by running all the tasks in Taskcluster Anyway for now my job is to get the firefox ui tests running in Taskcluster once a build task has been finished Although that this can only be done for Linux right now it shouldn t matter that much given that nothing in our firefox puppeteer package is platform dependent so far Expanding testing to other platforms should be trivial later on For now the primary goal is to see test results of our tests in Treeherder and letting developers know what needs to be changed if e g UI changes are causing a regression for us If you are interested in more details have a look at bug 1237550 Documentation of firefox ui tests and mozmill ci We are submitting our test results to Treeherder for a while and are pretty stable But the jobs are still listed as Tier 3 and are not taking care of by sheriffs To reach the Tier 2 level we definitely need proper documentation for our firefox ui tests and especially mozmill ci In case of test failures or build bustage the sheriffs have to know what s necessary to do Now that the dust caused by all the refactoring and moving the firefox ui tests to hg mozilla org settles a bit we want to start to work more with contributors again To allow an easy contribution I will create various project documentation which will show how to get started and how to submit patches Ultimately I want to see a quarter of contribution project for our firefox ui tests around mid this year Lets see how this goes More details about that can be found on bug 1237552 automation ci continuous integration firefox marionette QA Misc Mozilla Review of automation work Q4 2015 January 9 2016 Henrik Skupin 2 Comments The last quarter of 2015 is gone and its time to reflect what happened in Q4 In the following you will find a full overview again for the whole quarter It will be the last time that I will do that From now on I will post in shorter intervals to specific topics instead of covering everything This was actually a wish from our latest automation survey which I want to implement now I hope you will like it So during the last quarter my focus was completely on getting our firefox ui tests moved into mozilla central and to use mozharness to execute firefox ui tests in mozmill ci via the test archive As result I had lesser time for any other project So lets give some details Firefox UI Tests Mozharness One thing you really want to have with tests located in the tree is that those are not failing So I spent a good amount of time to fix our top failures and all those regressions as caused by UI changes like the security center in Firefox as preparation for the move I got them all green and try my best to keep that state now while we are in the transition The next thing was to clean up the repository and split apart all the different sub folders into their own package With that others could e g depend on our firefox puppeteer package for their own tests The whole work of refactoring has been done on bug 1232967 If you wonder why this bug is not closed yet it s because we still have to wait with the landing of the patch until mozmill ci production uses the new mozharness code This will hopefully happen soon and only wait of some other bugs to be fixed But based on those created packages we were able to use exactly that code to get our harness puppeteer and tests landed on http hg mozilla org mozilla central We also package them into the common tests zip archive for use in mozmill ci Details about all that can be found on bug 1212609 But please be aware that we still use the Github repository as integration repository I regularly mirror the code to hg which has to happen until we can also use the test package for localized builds and update tests Beside all that there were also a couple of mozharness fixes necessary So I implemented a better fetching of the tooltool script added the uninstall feature and also setup the handling of crash symbols for firefox ui tests Finally the addition of test package support finished up my work on mozharness for Q4 in 2015 During all the time I was also sheriffing our test results on Treeherder e g mozilla central because we are still Tier 3 level and sheriffs don t care about it Mozmill CI Our Jenkins based CI system is still called mozmill ci even it doesn t really run any mozmill tests anymore We decided to not change its name given that it will only be around this year until we can run all of our tests in TaskCluster But lots of changes have been landed which I want to announce below We followed Release Engineering and got rid of the OS X 10 8 testing That means the used Mac minis were ready to get re imaged with OS X 10 11 The transition worked seamlessly Enhancements for test report submission to Treeherder by switching over to Hawk credentials and more understandable group names and symbols Preparation of all the machines of our supported platforms OSX 10 6 10 9 10 10 10 11 Ubuntu 14 04 Windows XP 7 8 1 to be able to handle mozharness driven tests Implemented and updated scripts to run firefox ui tests via mozharness and to allow Treeherder to fully parse our logs Addons Tools I also had some time to work on supporting tools Together with the help of contributors we got the following done mozdownload Release of mozdownload 1 18 1 and 1 19 to cope with the move of builds to AWS Nightly Tester Tools Porting back changes from the NTT fork called Utilu Nightly Tester Tools Memchaser Thanks to the tireless efforts from Szabolcs Hubai we got Memchaser working again in latest Firefox builds So all in all it was a productive quarter with lots of things accomplished I m glad that we got all of this done Now in Q1 it will continue and more interesting work is in front of me which I m excited about I will announce that soon in my next blog post Until then I would like to give a little more insight into our current core team for Firefox automation A picture taken during our all hands work week in Orlando early in December shows Syd Maja myself and David Lets get started into 2016 with lots of ideas discussions and enough energy to get those things done automation ci continuous integration firefox jenkins marionette mozqa Mozilla Firefox Automation report Q3 2015 October 20 2015 Henrik Skupin 2 Comments It s time for another Firefox Automation report It s incredible how fast a quarter passes by without that I have time to write reports more often Hopefully it will change soon news will be posted in a follow up blog post Ok so what happened last quarter for our projects Mozharness One of my deliverables in Q3 was to create mozharness scripts for our various tests in the firefox ui tests repository so that our custom runner scripts can be replaced This gives us a way more stable system and additional features like crash report handling which are necessary to reach the tier 2 level in Treeherder After some refactoring of the firefox ui tests scripts for the functional and update tests were needed But before those could be implemented I had to spent some time in refactoring some modules of mozharness to make them better configurable for non buildbot jobs All that worked pretty fine and finally the entry scripts have been written Something special for them is that they even have different customers so extra configuration files had to be placed In detail it s us who run the tests in Jenkins for nightly builds and partly for release builds On the other side Release Engineering want to run our update tests on their own hardware when releases have to be tested By the end of September all work has been finished If you are interested in more details feel free to check the tracking bug 1192369 Mozmill CI Our Jenkins instance got lots of updates for various new features and necessary changes All in all I pushed 27 commits which affected 53 files Here a list of the major changes Refactoring of the test jobs has been started so that those can be used for mozharness driven firefox ui tests later in Q4 The work has not been finished and will be continued in Q4 Especially the refactoring for report submission to Treeherder even for aborted builds will be a large change A lot of time had to be spent in fixing the update tests for all the changes which were coming in with the Funsize project of Release Engineering Due to missing properties in the Mozilla Pulse messages update tests could no longer be triggered for nightly builds Therefore the handling of Pulse messages has been completely rewritten to allow the handling of similar Pulse messages as sent out from TaskCluster That work was actually not planned and has been stolen me quite some time from other projects A separation of functional and remote tests didn t make that much sense Especially because both types are actually functional tests As result they have been merged together into the functional tests You can still run remote tests only by using tag remote similar for tests with local testcases by using tag local We stopped running tests for mozilla esr31 builds due to Firefox ESR31 is no longer supported To lower the amount of machines we have to maintain and to getting closer what s being run on Buildbot we stopped running tests on Ubuntu 14 10 Means we only run on Ubuntu LTS releases from now on Also we stopped tests for OS X 10 8 The nodes will be re used for OS X 10 11 once released We experienced Java crashes due to low memory conditions of our Jenkins production master again This was kinda critical because the server is not getting restarted automatically After some investigation I assumed that the problem is due to the 32bit architecture of the VM Given that it has 8GB of memory a 64bit version of Ubuntu should have been better used So we replaced the machine and so far everything looks fine Totally surprising we had to release once more a bugfix release of Mozmill This time the framework didn t work at all due to the enforcement of add on signing So Mozmill 2 0 10 2 has been released automation continuous integration firefox jenkins marionette mozmill Software Mozilla mozdownload 1 18 released September 14 2015 Henrik Skupin 4 Comments Today we have released mozdownload 1 18 to PyPI The reason why I think it s worth a blog post is that with this version we finally added support for a sane API With it available using the mozdownload code in your own script is getting much easier So there is no need to instantiate a specific scraper anymore but a factory scraper is doing all the work depending on the options it gets Here some examples from mozdownload import FactoryScraper scraper FactoryScraper release version 40 0 3 locale de scraper download from mozdownload import FactoryScraper scraper FactoryScraper candidate version 41 0b9 platform win32 scraper download from mozdownload import FactoryScraper scraper FactoryScraper daily branch mozilla aurora scraper download If you are using mozdownload via its API you can also easily get the remote URL and the local filename from mozdownload import FactoryScraper scraper FactoryScraper daily branch mozilla aurora print scraper url print scraper filename Hereby the factory class is smart enough to only select those passed in options which are appropriate for the type of scraper If you have to download different types of builds you will enjoy that feature given that only the scraper type has to be changed and all other options could be still passed in We hope that this new feature will help you by integrating mozdownload into your own project There is no need anymore by using its command line interface through a subprocess call The complete changelog can be found here automation firefox mozdownload QA Software Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and finally a massive loss of core members Which means from the former 6 people only myself are remaining Since end of February all 5 former team members from Softvision are no longer participating in any of the maintained projects Thanks to all of them for the great help over all the last months and years But every project we own is now on my own shoulders And this is kinda hell of work with downsides like not being able to do as many reviews as I want for side projects One positive thing at least was that I got pulled back into the A Team at the same time With that move I m once more closer again to all the people who care about the basics of all test infrastructure at Mozilla I feel back home So what have I done the whole last quarter First it was always the ongoing daily work for maintaining our Mozmill CI system This was usually a job for a dedicated person all the last months The amount of work can sometimes eat up a whole day Especially if several regressions have been found or incompatible changes in Firefox have been landed Seeing my deliverables for Q1 it was clear that we have to cut down the time to spent on those failures As result we started to partially skip tests which were failing There was no time to get any of those fixed Happily the latest version of Mozmill is still working kinda nicely so no other work had to be dedicated for this project Most of my time during the last quarter I actually had to spent on Marionette especially building up wrapper scripts for being able to use Marionette as test framework for Firefox Desktop This was a kinda large change for us but totally important in terms of maintenance burden and sustainability The code base of Mozmill is kinda outdated and features like Electrolysis e10s will totally break it Given that a rewrite of the test framework is too cost intensive the decision has been made to transition our Mozmill tests over to Marionette Side effect was that a lot of missing features had to be implemented in Marionette to bring it at a level as what Mozmill offers Thanks for the amazing work goes to Andrew Halberstadt David Burns Jonathan Griffin and especially Chris Manchester For the new UI driven tests for Firefox Desktop we created the firefox ui tests repository at Github We decided on that name to make it clear to which product the tests belong to and also to get rid of any relationship to the underling test framework name This repository contains the harness extensions around Marionette a separate puppeteer library for back end and UI modules and last but not least the tests themselves As goal for Q1 we had to get the basics working including the full set of remote security tests and most important the update tests A lot of help on the security tests we got from Barbara Miller our intern from December to March She did great amount of work here and also assisted other community members in getting their code done Finally we got all the security tests converted My own focus beside the harness pieces were the update tests Given the complete refactoring of those Mozmill tests we were able to easily port them over to Marionette We tried to keep the class structure as is and only did enhancements where necessary Here Bob Silverberg helped with two big chunks of work which I m gladly thankful about Thanks a lot With all modules in place I finally converted the update tests and got them running for each version of Firefox down to 38 0 which will be the next ESR release and kinda important to be tested with Marionette For stability and ease of contribution we added support for Travis CI to our new repository It helps us a lot with reviews of patches from community members and they also can see immediately if changes they have done are working as

    Original URL path: http://www.hskupin.info/tag/firefox/ (2016-05-01)
    Open archived version from archive

  • marionette | hskupin.info
    1237552 automation ci continuous integration firefox marionette QA Misc Mozilla Review of automation work Q4 2015 January 9 2016 Henrik Skupin 2 Comments The last quarter of 2015 is gone and its time to reflect what happened in Q4 In the following you will find a full overview again for the whole quarter It will be the last time that I will do that From now on I will post in shorter intervals to specific topics instead of covering everything This was actually a wish from our latest automation survey which I want to implement now I hope you will like it So during the last quarter my focus was completely on getting our firefox ui tests moved into mozilla central and to use mozharness to execute firefox ui tests in mozmill ci via the test archive As result I had lesser time for any other project So lets give some details Firefox UI Tests Mozharness One thing you really want to have with tests located in the tree is that those are not failing So I spent a good amount of time to fix our top failures and all those regressions as caused by UI changes like the security center in Firefox as preparation for the move I got them all green and try my best to keep that state now while we are in the transition The next thing was to clean up the repository and split apart all the different sub folders into their own package With that others could e g depend on our firefox puppeteer package for their own tests The whole work of refactoring has been done on bug 1232967 If you wonder why this bug is not closed yet it s because we still have to wait with the landing of the patch until mozmill ci production uses the new mozharness code This will hopefully happen soon and only wait of some other bugs to be fixed But based on those created packages we were able to use exactly that code to get our harness puppeteer and tests landed on http hg mozilla org mozilla central We also package them into the common tests zip archive for use in mozmill ci Details about all that can be found on bug 1212609 But please be aware that we still use the Github repository as integration repository I regularly mirror the code to hg which has to happen until we can also use the test package for localized builds and update tests Beside all that there were also a couple of mozharness fixes necessary So I implemented a better fetching of the tooltool script added the uninstall feature and also setup the handling of crash symbols for firefox ui tests Finally the addition of test package support finished up my work on mozharness for Q4 in 2015 During all the time I was also sheriffing our test results on Treeherder e g mozilla central because we are still Tier 3 level and sheriffs don t care about it Mozmill CI Our Jenkins based CI system is still called mozmill ci even it doesn t really run any mozmill tests anymore We decided to not change its name given that it will only be around this year until we can run all of our tests in TaskCluster But lots of changes have been landed which I want to announce below We followed Release Engineering and got rid of the OS X 10 8 testing That means the used Mac minis were ready to get re imaged with OS X 10 11 The transition worked seamlessly Enhancements for test report submission to Treeherder by switching over to Hawk credentials and more understandable group names and symbols Preparation of all the machines of our supported platforms OSX 10 6 10 9 10 10 10 11 Ubuntu 14 04 Windows XP 7 8 1 to be able to handle mozharness driven tests Implemented and updated scripts to run firefox ui tests via mozharness and to allow Treeherder to fully parse our logs Addons Tools I also had some time to work on supporting tools Together with the help of contributors we got the following done mozdownload Release of mozdownload 1 18 1 and 1 19 to cope with the move of builds to AWS Nightly Tester Tools Porting back changes from the NTT fork called Utilu Nightly Tester Tools Memchaser Thanks to the tireless efforts from Szabolcs Hubai we got Memchaser working again in latest Firefox builds So all in all it was a productive quarter with lots of things accomplished I m glad that we got all of this done Now in Q1 it will continue and more interesting work is in front of me which I m excited about I will announce that soon in my next blog post Until then I would like to give a little more insight into our current core team for Firefox automation A picture taken during our all hands work week in Orlando early in December shows Syd Maja myself and David Lets get started into 2016 with lots of ideas discussions and enough energy to get those things done automation ci continuous integration firefox jenkins marionette mozqa Mozilla Firefox Automation report Q3 2015 October 20 2015 Henrik Skupin 2 Comments It s time for another Firefox Automation report It s incredible how fast a quarter passes by without that I have time to write reports more often Hopefully it will change soon news will be posted in a follow up blog post Ok so what happened last quarter for our projects Mozharness One of my deliverables in Q3 was to create mozharness scripts for our various tests in the firefox ui tests repository so that our custom runner scripts can be replaced This gives us a way more stable system and additional features like crash report handling which are necessary to reach the tier 2 level in Treeherder After some refactoring of the firefox ui tests scripts for the functional and update tests were needed But before those could be implemented I had to spent some time in refactoring some modules of mozharness to make them better configurable for non buildbot jobs All that worked pretty fine and finally the entry scripts have been written Something special for them is that they even have different customers so extra configuration files had to be placed In detail it s us who run the tests in Jenkins for nightly builds and partly for release builds On the other side Release Engineering want to run our update tests on their own hardware when releases have to be tested By the end of September all work has been finished If you are interested in more details feel free to check the tracking bug 1192369 Mozmill CI Our Jenkins instance got lots of updates for various new features and necessary changes All in all I pushed 27 commits which affected 53 files Here a list of the major changes Refactoring of the test jobs has been started so that those can be used for mozharness driven firefox ui tests later in Q4 The work has not been finished and will be continued in Q4 Especially the refactoring for report submission to Treeherder even for aborted builds will be a large change A lot of time had to be spent in fixing the update tests for all the changes which were coming in with the Funsize project of Release Engineering Due to missing properties in the Mozilla Pulse messages update tests could no longer be triggered for nightly builds Therefore the handling of Pulse messages has been completely rewritten to allow the handling of similar Pulse messages as sent out from TaskCluster That work was actually not planned and has been stolen me quite some time from other projects A separation of functional and remote tests didn t make that much sense Especially because both types are actually functional tests As result they have been merged together into the functional tests You can still run remote tests only by using tag remote similar for tests with local testcases by using tag local We stopped running tests for mozilla esr31 builds due to Firefox ESR31 is no longer supported To lower the amount of machines we have to maintain and to getting closer what s being run on Buildbot we stopped running tests on Ubuntu 14 10 Means we only run on Ubuntu LTS releases from now on Also we stopped tests for OS X 10 8 The nodes will be re used for OS X 10 11 once released We experienced Java crashes due to low memory conditions of our Jenkins production master again This was kinda critical because the server is not getting restarted automatically After some investigation I assumed that the problem is due to the 32bit architecture of the VM Given that it has 8GB of memory a 64bit version of Ubuntu should have been better used So we replaced the machine and so far everything looks fine Totally surprising we had to release once more a bugfix release of Mozmill This time the framework didn t work at all due to the enforcement of add on signing So Mozmill 2 0 10 2 has been released automation continuous integration firefox jenkins marionette mozmill Software Mozilla Firefox Automation report Q2 2015 August 11 2015 Henrik Skupin 5 Comments It s been a while since I wrote my last Firefox automation report so lets do another one to wrap up everything happened in Q2 this year As you may remember from my last report the team has been cut down to only myself and beside that I was away the whole April Means only 2 months worth of work will be covered this time In general it was a tough quarter for me Working alone and having to maintain all of our infrastructure and keeping both of our tests Mozmill and Marionette green was more work as expected But it still gave me enough time to finish my two deliverables Ensure better visibility of Firefox UI test results for sheriffs and developers by uploading them to treeherder Finalize features for Firefox UI update tests and help to get them running on RelEng hardware Firefox UI Test Results on Treeherder With the transition of Mozmill tests to Marionette we no longer needed the mozmill dashboard Especially not since nearly no job in our Mozmill CI is running Mozmill anymore Given that we also weren t happy with the dashboard solution and the usage of couchdb under the hood I was happy that a great replacement exist and our Marionette tests can make use of Treeherder is the new reporting system for tests being run in buildbot continuous integration Because of that it covers all products and their appropriate supported branches That s why it it s kinda perfect for us Before I was able to get started a bit of investigation had to be done and I also checked some other projects which make use of treeherder reporting That information was kinda helpful even with lots of undocumented code Given that our tests are located outside of the development tree in its own Github repository some integration can be handled more loosely compared to in tree tests With that respect we do not appear as Tier 1 or Tier 2 but results are listed under the Tier 3 level which is hidden by default But that s fine given that our goal was to bring up reports to treeherder first Going into details will be a follow up task For reporting results to Treeherder the Python package treeherder client can be used It s a collection of different classes which help with authentication collecting job details and finally uploading the data to the server It s documentation can be found on readthedocs and meanwhile contains a good number of example code which makes it easier to get your code implemented Before you can actually upload reports the names and symbols of the groups and jobs have to be defined For our tests I have chosen the three group symbols Fu Ff and Fr Each of those stays for F irefox UI Tests and the first letter of the testrun name We currently have updates functional and remote As job name the locale of Firefox will be used That results in an output like the following Whether jobs are passing or failing it is recommended to always add the log files as generated by running the tests to the report It will not happen via data URLs but as artifacts with a link to an upload location In our case we make use of Amazon S3 as storage service With all the pieces implemented we can now report to treeherder and covering Nightly Aurora Developer Edition Beta and Release builds As of now reporting only happens against the staging instance of Treeherder but soon we will be able to report to production If you want to have a sneak peak how it works just follow this link for Nightly builds More details about Treeherder reporting I will do later this quarter when the next pieces have been implemented Firefox UI Update Tests under RelEng My second deliverable was to assist Armen Zambrano in getting the Firefox UI Update tests run for beta and release builds on Release Engineering infrastructure This was a kinda important goal for us given that until now it was a manually triggered process with lots of human errors on our own infrastructure That means lots of failures if you do not correctly setup the configuration for the tests and a slower processing of builds due to our limited available infrastructure So moving this out of our area made total sense Given that Armen had already done a fair amount of work when I came back from my PTO I majorly fixed issues for the tests and the libraries as pointed out by him All that allowed us to finally run our tests on Release Engineering infrastructure even with a couple of failures at the beginning for the first beta But those were smaller issues and got fixed quickly Since then we seem to have good results If you want to have a look in how that works you should check the Marionette update tests wiki page Sadly some of the requirements haven t been completely finished yet So the Quality Engineering team cannot stop running the tests themselves But that will happen once bug 1182796 has been fixed and deployed to production Oh and if you wonder where the results are located Those are not getting sent to Treeherder but to an internal mailing list as used for every other automation results Other Work Beside the deliverables I got some more work done Mainly for the firefox ui tests and mozmill ci While the test coverage has not really been increased I had a couple of regressions to fix as caused by changes in Firefox But we also landed some new features thankfully as contributed by community members Once all that was done and we agreed to have kinda stable tests new branches have been created in the repository That was necessary to add support for each supported version of Firefox down to ESR 38 0 and to be able to run the tests in our Mozmill CI via Jenkins More about that you will find below The only task I still haven t had time for yet was the creation of proper documentation about our tests I hope that I will find the time in Q3 Mozmill CI got the most changes in Q2 compared to all the former quarters This is related to the transition from Mozmill tests to Marionette tests More details why we got rid of Mozmill tests can be found in this post With that we decided to get rid of most of the tests and mainly start from scratch by only porting the security and update tests over to Marionette The complete replacement in Mozmill and all its jobs can be seen on issue 576 on Github In detail we have the following major changes Run all jobs with Marionette beside Firefox ESR 31 0 which is not supported by Marionette and ondemand update jobs because they still have to be run by Quality Engineering Reduced number of platforms We got rid of Windows Vista Ubuntu 14 10 and OS X 10 7 whereby the latter machines have been re used for OS X 10 10 No usage of a pre configured environments anymore but creating it from fresh for each test run by installing Python packages from the internal PyPI mirror Sending test results to treeherder and giving public access for everyone Stopped sending emails for failures to our mozmill ci mailing list in favor of having treeherder results All changes in Mozmill CI can be seen on Github Last but not least we also had two releases of mozdownload in Q2 Both had a good amount of features included For details you can check the changelog I hope that gave you a good quick read on the stuff I was working on last quarter Maybe in Q3 I will find the time to blog more often and in more detail Lets see automation ci continuous integration jenkins marionette mozmill Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and

    Original URL path: http://www.hskupin.info/tag/marionette/ (2016-05-01)
    Open archived version from archive

  • mozqa | hskupin.info
    promotion we create the candidates based on a signed off CI build we already have a valid revision to be used with mozdownload to retrieve the test packages json file so no need for the above mentioned Treeherder traversal code o Once all has been implemented Firefox 46 0b3 was the first beta release for which we were able to process the release promotion notifications At the same time with release promotion news I also got informed by Robert Kaiser that the ondemand update jobs as performed with Mozmill do not work anymore As turned out a change in the JS engine caused the bustage for Firefox 46 0b1 Given that Mozmill is dead I was not going to update it again Instead I converted the ondemand update jobs to make use of Firefox UI Tests This went pretty well also because we were running those tests already for a while on mozilla central and mozilla aurora for nightly builds As result we were able to run update jobs a day later for Firefox 46 0b1 and noticed that nearly all locales on Windows were busted so only en US got finally shipped Not sure if that would have been that visible with Mozmill Last but not least I also removed the workaround which let all test jobs use the mozharness script from mozilla central It s simply not necessary anymore given that all required features in mozharness are part of ESR45 now What s next I already have plans what s next But given that I will be away from work for a full month now I will have to revisit those once I m back in May I promise that I will also blog about them around that time automation ci firefox marionette mozqa QA testing Misc Mozilla Review of automation work Q4 2015 January 9 2016 Henrik Skupin 2 Comments The last quarter of 2015 is gone and its time to reflect what happened in Q4 In the following you will find a full overview again for the whole quarter It will be the last time that I will do that From now on I will post in shorter intervals to specific topics instead of covering everything This was actually a wish from our latest automation survey which I want to implement now I hope you will like it So during the last quarter my focus was completely on getting our firefox ui tests moved into mozilla central and to use mozharness to execute firefox ui tests in mozmill ci via the test archive As result I had lesser time for any other project So lets give some details Firefox UI Tests Mozharness One thing you really want to have with tests located in the tree is that those are not failing So I spent a good amount of time to fix our top failures and all those regressions as caused by UI changes like the security center in Firefox as preparation for the move I got them all green and try my best to keep that state now while we are in the transition The next thing was to clean up the repository and split apart all the different sub folders into their own package With that others could e g depend on our firefox puppeteer package for their own tests The whole work of refactoring has been done on bug 1232967 If you wonder why this bug is not closed yet it s because we still have to wait with the landing of the patch until mozmill ci production uses the new mozharness code This will hopefully happen soon and only wait of some other bugs to be fixed But based on those created packages we were able to use exactly that code to get our harness puppeteer and tests landed on http hg mozilla org mozilla central We also package them into the common tests zip archive for use in mozmill ci Details about all that can be found on bug 1212609 But please be aware that we still use the Github repository as integration repository I regularly mirror the code to hg which has to happen until we can also use the test package for localized builds and update tests Beside all that there were also a couple of mozharness fixes necessary So I implemented a better fetching of the tooltool script added the uninstall feature and also setup the handling of crash symbols for firefox ui tests Finally the addition of test package support finished up my work on mozharness for Q4 in 2015 During all the time I was also sheriffing our test results on Treeherder e g mozilla central because we are still Tier 3 level and sheriffs don t care about it Mozmill CI Our Jenkins based CI system is still called mozmill ci even it doesn t really run any mozmill tests anymore We decided to not change its name given that it will only be around this year until we can run all of our tests in TaskCluster But lots of changes have been landed which I want to announce below We followed Release Engineering and got rid of the OS X 10 8 testing That means the used Mac minis were ready to get re imaged with OS X 10 11 The transition worked seamlessly Enhancements for test report submission to Treeherder by switching over to Hawk credentials and more understandable group names and symbols Preparation of all the machines of our supported platforms OSX 10 6 10 9 10 10 10 11 Ubuntu 14 04 Windows XP 7 8 1 to be able to handle mozharness driven tests Implemented and updated scripts to run firefox ui tests via mozharness and to allow Treeherder to fully parse our logs Addons Tools I also had some time to work on supporting tools Together with the help of contributors we got the following done mozdownload Release of mozdownload 1 18 1 and 1 19 to cope with the move of builds to AWS Nightly Tester Tools Porting back changes from the NTT fork called Utilu Nightly Tester Tools Memchaser Thanks to the tireless efforts from Szabolcs Hubai we got Memchaser working again in latest Firefox builds So all in all it was a productive quarter with lots of things accomplished I m glad that we got all of this done Now in Q1 it will continue and more interesting work is in front of me which I m excited about I will announce that soon in my next blog post Until then I would like to give a little more insight into our current core team for Firefox automation A picture taken during our all hands work week in Orlando early in December shows Syd Maja myself and David Lets get started into 2016 with lots of ideas discussions and enough energy to get those things done automation ci continuous integration firefox jenkins marionette mozqa Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and finally a massive loss of core members Which means from the former 6 people only myself are remaining Since end of February all 5 former team members from Softvision are no longer participating in any of the maintained projects Thanks to all of them for the great help over all the last months and years But every project we own is now on my own shoulders And this is kinda hell of work with downsides like not being able to do as many reviews as I want for side projects One positive thing at least was that I got pulled back into the A Team at the same time With that move I m once more closer again to all the people who care about the basics of all test infrastructure at Mozilla I feel back home So what have I done the whole last quarter First it was always the ongoing daily work for maintaining our Mozmill CI system This was usually a job for a dedicated person all the last months The amount of work can sometimes eat up a whole day Especially if several regressions have been found or incompatible changes in Firefox have been landed Seeing my deliverables for Q1 it was clear that we have to cut down the time to spent on those failures As result we started to partially skip tests which were failing There was no time to get any of those fixed Happily the latest version of Mozmill is still working kinda nicely so no other work had to be dedicated for this project Most of my time during the last quarter I actually had to spent on Marionette especially building up wrapper scripts for being able to use Marionette as test framework for Firefox Desktop This was a kinda large change for us but totally important in terms of maintenance burden and sustainability The code base of Mozmill is kinda outdated and features like Electrolysis e10s will totally break it Given that a rewrite of the test framework is too cost intensive the decision has been made to transition our Mozmill tests over to Marionette Side effect was that a lot of missing features had to be implemented in Marionette to bring it at a level as what Mozmill offers Thanks for the amazing work goes to Andrew Halberstadt David Burns Jonathan Griffin and especially Chris Manchester For the new UI driven tests for Firefox Desktop we created the firefox ui tests repository at Github We decided on that name to make it clear to which product the tests belong to and also to get rid of any relationship to the underling test framework name This repository contains the harness extensions around Marionette a separate puppeteer library for back end and UI modules and last but not least the tests themselves As goal for Q1 we had to get the basics working including the full set of remote security tests and most important the update tests A lot of help on the security tests we got from Barbara Miller our intern from December to March She did great amount of work here and also assisted other community members in getting their code done Finally we got all the security tests converted My own focus beside the harness pieces were the update tests Given the complete refactoring of those Mozmill tests we were able to easily port them over to Marionette We tried to keep the class structure as is and only did enhancements where necessary Here Bob Silverberg helped with two big chunks of work which I m gladly thankful about Thanks a lot With all modules in place I finally converted the update tests and got them running for each version of Firefox down to 38 0 which will be the next ESR release and kinda important to be tested with Marionette For stability and ease of contribution we added support for Travis CI to our new repository It helps us a lot with reviews of patches from community members and they also can see immediately if changes they have done are working as expected The next big chunk of work will be to get those tests running in Mozmill CI to be renamed and the test reporting to use Treeherder Also we want to get our update tests for Firefox releases executed by the RelEng system to further reduce the amount of time for signoffs from QE About this work I will talk more in my next blog post So please stay tuned Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agendas the video recordings and notes from the Firefox Automation meetings Please note that since end of February we no longer host a meeting due to the low attendance and other meetings like the A team ones where I have to report my status automation continuous integration firefox marionette mozmill mozqa QA testing Mozilla Firefox Automation report week 51 52 2014 March 2 2015 Henrik Skupin 3 Comments In this post you can find an overview about the work happened in the Firefox Automation team during week 51 and 52 of 2014 I m sorry for this very late post but changes to our team which I will get to in my next upcoming post caught me up with lots of more work and didn t give me the time for writing status reports Highlights Henrik started work towards a Mozmill 2 1 release Therefore he had to upgrade a couple of mozbase packages first to get latest Mozmill code on master working again Once done the patch for handling parent sections in manifest files finally landed which was originally written by Andrei Eftimie and was sitting around for a while That addition allows us to use mozhttpd for serving test data via a local HTTP server Last but not least another important feature went in which let us better handle application disconnects There are still some more bugs to fix before we can actually release version 2 1 of Mozmill Given that we only have the capacity to fix the most important issues for the Mozmill test framework Henrik started to mass close existing bugs for Mozmill So only a hand full of bugs will remain open If there is something important you want to see fixed we would encourage you to start working on the appropriate bug For Mozmill CI we got the new Ubuntu 14 10 boxes up and running in our staging environment Once we can be sure they are stable enough they will also be enabled in production Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 51 and week 52 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meeting of week 51 and week 52 automation ci firefox marionette mozmill mozqa testing Mozilla Firefox Automation report week 49 50 2014 January 22 2015 Henrik Skupin 5 Comments In this post you can find an overview about the work happened in the Firefox Automation team during week 49 and 50 Highlights During the first week of December the all hands work week happened in Portland Those were some great and inspiring days full of talks discussions and conversations about various things Given that I do not see my colleagues that often in real life I have taken this opportunity to talk to everyone who is partly or fully involved in projects of our automation team There are various big goals in front of us so clearing questions and finding the next steps to tackle ongoing problems was really important Finally we came out with a long list of todo items and more clarity about so far unclear tasks In week 50 we got some updates landed for Mozmill CI Given a regression from the blacklist landing our l10n tests haven t been executed for any locale of the Firefox Developer Edition Since the fix landed we have seen problems with access keys in nearly each locale for a new test which covers the context menu of web content Also we would like to welcome Barbara Miller in our team She joined us as an intern via the FOSS outreach program as driven by Gnome She will be with us until March and will mainly work on testdaybot and the conversion of Mozmill tests to Marionette The latter project is called m21s and details can be found on its project page Soon I will post more details about it Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 49 and week 50 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meeting of week 48 Due to the Mozilla all hands workweek there was no meeting in week 49 automation continuous integration marionette mozmill mozqa testing Mozilla Software Memchaser 0 6 has been released September 18 2014 Henrik Skupin 2 Comments The Firefox Automation team would like to announce the release of memchaser 0 6 After nearly a year of no real feature updates but also some weeks of not being able to run Memchaser in Firefox Aurora 34 0a2 at all due to a regression we decided to release the current state of development as a public release We are aware that we still do not fully support the default Australis theme since Firefox 29 0 but that s an issue which takes some more time to finish up Changes in 0 6 Upgrade to Add on SDK 1 17 201 Fix test start stop logging for File not found error 199 contentURL expects a string in Panel and Widget constructors 198 Bring the minimizeMemory implementation up to date 193 Use residentFast instead of resident for memory reporter 192 Use postMessage for widget and panel communication instead port object 185 Support incremental cycle collection statistics 187 Added a usage section to the README 178 Change require statements and update the SDK to 1 14 174 Simplify travis yml 172 Use dot to include files while invoking bin sh 169

    Original URL path: http://www.hskupin.info/tag/mozqa/ (2016-05-01)
    Open archived version from archive

  • QA | hskupin.info
    has been implemented Firefox 46 0b3 was the first beta release for which we were able to process the release promotion notifications At the same time with release promotion news I also got informed by Robert Kaiser that the ondemand update jobs as performed with Mozmill do not work anymore As turned out a change in the JS engine caused the bustage for Firefox 46 0b1 Given that Mozmill is dead I was not going to update it again Instead I converted the ondemand update jobs to make use of Firefox UI Tests This went pretty well also because we were running those tests already for a while on mozilla central and mozilla aurora for nightly builds As result we were able to run update jobs a day later for Firefox 46 0b1 and noticed that nearly all locales on Windows were busted so only en US got finally shipped Not sure if that would have been that visible with Mozmill Last but not least I also removed the workaround which let all test jobs use the mozharness script from mozilla central It s simply not necessary anymore given that all required features in mozharness are part of ESR45 now What s next I already have plans what s next But given that I will be away from work for a full month now I will have to revisit those once I m back in May I promise that I will also blog about them around that time automation ci firefox marionette mozqa QA testing Mozilla Firefox Desktop automation goals Q1 2016 February 2 2016 Henrik Skupin 2 Comments As promised in my last blog posts I don t want to only blog about the goals from last quarters but also about planned work and what s currently in progress So this post will be the first one which will shed some light into my active work First lets get started with my goals for this quarter Execute firefox ui tests in TaskCluster Now that our tests are located in mozilla central mozilla aurora and mozilla beta we want to see them run on a check in basis including try Usually you will setup Buildbot jobs to get your wanted tasks running But given that the build system will be moved to Taskcluster in the next couple of months we decided to start directly with the new CI infrastructure So how will this look like and how will mozmill ci cope with that For the latter I can say that we don t want to run more tests as we do right now This is mostly due to our limited infrastructure I have to maintain myself Having the needs to run firefox ui tests for each check in on all platforms and even for try pushes would mean that we totally exceed the machine capacity Therefore we continue to use mozmill ci for now to test nightly and release builds for en US but also a couple of other locales This might change later this year when mozmill ci can be replaced by running all the tasks in Taskcluster Anyway for now my job is to get the firefox ui tests running in Taskcluster once a build task has been finished Although that this can only be done for Linux right now it shouldn t matter that much given that nothing in our firefox puppeteer package is platform dependent so far Expanding testing to other platforms should be trivial later on For now the primary goal is to see test results of our tests in Treeherder and letting developers know what needs to be changed if e g UI changes are causing a regression for us If you are interested in more details have a look at bug 1237550 Documentation of firefox ui tests and mozmill ci We are submitting our test results to Treeherder for a while and are pretty stable But the jobs are still listed as Tier 3 and are not taking care of by sheriffs To reach the Tier 2 level we definitely need proper documentation for our firefox ui tests and especially mozmill ci In case of test failures or build bustage the sheriffs have to know what s necessary to do Now that the dust caused by all the refactoring and moving the firefox ui tests to hg mozilla org settles a bit we want to start to work more with contributors again To allow an easy contribution I will create various project documentation which will show how to get started and how to submit patches Ultimately I want to see a quarter of contribution project for our firefox ui tests around mid this year Lets see how this goes More details about that can be found on bug 1237552 automation ci continuous integration firefox marionette QA Mozilla Automation Survey Follow up January 5 2016 Henrik Skupin 2 Comments As promised in my last post about the automation survey results I wanted to come up with a follow up to clarify our next steps in being more open for our activities discussions and also quarterly goals Sorry that it has been taken a bit longer but end of the quarter and especially the year is mostly packed with stuff to finish up Also the all hands work week in Orlando beginning of December hold me off from doing a lot real work So lets get started with the mailing list topic first As we have seen most people kinda like to get our news via the automation mailing list But given the low usage of that list in the last months it was a bit surprising Nearly all the time I sent emails myself not to count in Travis results That means we want to implement a change here From now on we won t use the mozilla dev automation list but instead utilize the mozilla tools list Also because this is the recommended list for the Engineering Productivity team we are all part of and discussions will reach a larger audience So please subscribe to this list via Google Groups or Email For status updates about our current activities we started to use standu ps last quarter It seems to work pretty well for us and everyone else is welcome to also post updates to our automation project section If you are interested in those updates then read through that list or simply subscribe the page in your RSS reader Please also note that from now on there will be no Firefox Automation reports anymore Instead I will reduce the amount of different contents and only write about projects I worked on So keep an eye out to not miss those automation Mozilla QA Mozilla Results of the Firefox Automation Survey December 3 2015 Henrik Skupin 2 Comments November 23rd I blogged about the active survey covering the information flow inside our Firefox Automation team This survey was open until November 30th and I thank everyone of the participants which have taken the time to get it filled out In the following you can find the results Most of the contributors who are following our activities are with Mozilla for the last 3 years Whereby half of them joined less than a year ago There is also a 1 1 split between volunteers and paid staff members This is most likely because of the low number of responses but anyway increasing the number of volunteers is certainly something we want to follow up on in the next months The question about which communication channel is preferred to get the latest news got answered with 78 for the automation mailing list I feel that this is a strange result given that we haven t really used that list for active discussions or similar in the past months But that means we should put more focus on the list Beside that also 55 listening our activities on Bugzilla via component watchers I would assume that those people are mostly our paid staff who kinda have to follow each others work regarding reviews needinfo requests and process updates 44 of all read our blog posts on the Mozilla A Team Planet So we will put more focus in the future to both blog posts and discussions on the mailing list More than half of our followers check for updates at least once a day So when we get started with interesting discussions I would expect good activity throughout the day 44 of all feel less informed about our current activities Another 33 answered this question with Mostly So it s a clear indication what I already thought and which clearly needs action on our side to be more communicative Doing this might also bring more people into our active projects so mentoring would be much more valuable and time effective as handling any drive by projects which we cannot fully support A request for the type of news we should do more is definitely for latest changes and code landings from contributors This will ensure people feel recognized and contributors will also know each others work and see the effectiveness in regards of our project goals But also discussions about various automation related topics as mentioned already above are highly wanted Other topics like quarterly goals and current status updates are also wanted and we will see how we can do that We might be able to fold those general updates into the Engineering Productivity updates which are pushed out twice a month via the A Team Planet Also there is a bit of confusion about the Firefox Automation team and how it relates to the Engineering Productivity team formerly A Team Effectively we are all part of the latter and the virtual Automation team has only been created when we got shifted between the A Team and QA Team forth and back This will not happen anymore so we agreed on to get rid of this name All in all there are some topics which will need further discussions I will follow up with another blog post soon which will show off our plans for improvements and how we want to work to make it happen automation QA survey Mozilla Survey about sharing information inside the Firefox Automation team November 23 2015 Henrik Skupin 2 Comments Within the Firefox Automation team we were suffering a bit in sharing information about our work over the last couple of months That mainly happened because I was alone and not able to blog more often than once in a quarter The same applies to our dev automation mailing list which mostly only received emails from Travis CI with testing results Given that the team has been increased to 4 people now beside me this is Maja Frydrychowicz Syd Polk and David Burns we want to be more open again and also trying to get more people involved into our projects To ensure that we do not make use of the wrong communication channels depending where most of our readers are I have setup a little survey It will only take you a minute to go through but it will help us a lot to know more about the preferences of our automation geeks So please take that little time and help us The survey can be found here and is open until end of November 2015 https www surveymonkey com r 528WYYJ Thank you a lot automation Mozilla QA survey testing Mozilla mozdownload 1 18 released September 14 2015 Henrik Skupin 4 Comments Today we have released mozdownload 1 18 to PyPI The reason why I think it s worth a blog post is that with this version we finally added support for a sane API With it available using the mozdownload code in your own script is getting much easier So there is no need to instantiate a specific scraper anymore but a factory scraper is doing all the work depending on the options it gets Here some examples from mozdownload import FactoryScraper scraper FactoryScraper release version 40 0 3 locale de scraper download from mozdownload import FactoryScraper scraper FactoryScraper candidate version 41 0b9 platform win32 scraper download from mozdownload import FactoryScraper scraper FactoryScraper daily branch mozilla aurora scraper download If you are using mozdownload via its API you can also easily get the remote URL and the local filename from mozdownload import FactoryScraper scraper FactoryScraper daily branch mozilla aurora print scraper url print scraper filename Hereby the factory class is smart enough to only select those passed in options which are appropriate for the type of scraper If you have to download different types of builds you will enjoy that feature given that only the scraper type has to be changed and all other options could be still passed in We hope that this new feature will help you by integrating mozdownload into your own project There is no need anymore by using its command line interface through a subprocess call The complete changelog can be found here automation firefox mozdownload QA Software Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and finally a massive loss of core members Which means from the former 6 people only myself are remaining Since end of February all 5 former team members from Softvision are no longer participating in any of the maintained projects Thanks to all of them for the great help over all the last months and years But every project we own is now on my own shoulders And this is kinda hell of work with downsides like not being able to do as many reviews as I want for side projects One positive thing at least was that I got pulled back into the A Team at the same time With that move I m once more closer again to all the people who care about the basics of all test infrastructure at Mozilla I feel back home So what have I done the whole last quarter First it was always the ongoing daily work for maintaining our Mozmill CI system This was usually a job for a dedicated person all the last months The amount of work can sometimes eat up a whole day Especially if several regressions have been found or incompatible changes in Firefox have been landed Seeing my deliverables for Q1 it was clear that we have to cut down the time to spent on those failures As result we started to partially skip tests which were failing There was no time to get any of those fixed Happily the latest version of Mozmill is still working kinda nicely so no other work had to be dedicated for this project Most of my time during the last quarter I actually had to spent on Marionette especially building up wrapper scripts for being able to use Marionette as test framework for Firefox Desktop This was a kinda large change for us but totally important in terms of maintenance burden and sustainability The code base of Mozmill is kinda outdated and features like Electrolysis e10s will totally break it Given that a rewrite of the test framework is too cost intensive the decision has been made to transition our Mozmill tests over to Marionette Side effect was that a lot of missing features had to be implemented in Marionette to bring it at a level as what Mozmill offers Thanks for the amazing work goes to Andrew Halberstadt David Burns Jonathan Griffin and especially Chris Manchester For the new UI driven tests for Firefox Desktop we created the firefox ui tests repository at Github We decided on that name to make it clear to which product the tests belong to and also to get rid of any relationship to the underling test framework name This repository contains the harness extensions around Marionette a separate puppeteer library for back end and UI modules and last but not least the tests themselves As goal for Q1 we had to get the basics working including the full set of remote security tests and most important the update tests A lot of help on the security tests we got from Barbara Miller our intern from December to March She did great amount of work here and also assisted other community members in getting their code done Finally we got all the security tests converted My own focus beside the harness pieces were the update tests Given the complete refactoring of those Mozmill tests we were able to easily port them over to Marionette We tried to keep the class structure as is and only did enhancements where necessary Here Bob Silverberg helped with two big chunks of work which I m gladly thankful about Thanks a lot With all modules in place I finally converted the update tests and got them running for each version of Firefox down to 38 0 which will be the next ESR release and kinda important to be tested with Marionette For stability and ease of contribution we added support for Travis CI to our new repository It helps us a lot with reviews of patches from community members and they also can see immediately if changes they have done are working as expected The next big chunk of work will be to get those tests running in Mozmill CI to be renamed and the test reporting to use Treeherder Also we want to get our

    Original URL path: http://www.hskupin.info/tag/qa/ (2016-05-01)
    Open archived version from archive

  • testing | hskupin.info
    given platform allowed me to retrieve the next possible revision to be used with mozdownload to retrieve the test packages json URL I know its not perfect but satisfies us enough for now Then the release promotion project as worked on by the Release Engineering team was close to be activated I heard a couple of days before that Firefox 46 0b1 will be the first candidate to get it tested on It gave me basically no time for testing at all Thanks to all the support from Rail Aliiev I was able to get the new Mozilla Pulse listener created to handle appropriate release promotion build notifications Given that with release promotion we create the candidates based on a signed off CI build we already have a valid revision to be used with mozdownload to retrieve the test packages json file so no need for the above mentioned Treeherder traversal code o Once all has been implemented Firefox 46 0b3 was the first beta release for which we were able to process the release promotion notifications At the same time with release promotion news I also got informed by Robert Kaiser that the ondemand update jobs as performed with Mozmill do not work anymore As turned out a change in the JS engine caused the bustage for Firefox 46 0b1 Given that Mozmill is dead I was not going to update it again Instead I converted the ondemand update jobs to make use of Firefox UI Tests This went pretty well also because we were running those tests already for a while on mozilla central and mozilla aurora for nightly builds As result we were able to run update jobs a day later for Firefox 46 0b1 and noticed that nearly all locales on Windows were busted so only en US got finally shipped Not sure if that would have been that visible with Mozmill Last but not least I also removed the workaround which let all test jobs use the mozharness script from mozilla central It s simply not necessary anymore given that all required features in mozharness are part of ESR45 now What s next I already have plans what s next But given that I will be away from work for a full month now I will have to revisit those once I m back in May I promise that I will also blog about them around that time automation ci firefox marionette mozqa QA testing Mozilla Survey about sharing information inside the Firefox Automation team November 23 2015 Henrik Skupin 2 Comments Within the Firefox Automation team we were suffering a bit in sharing information about our work over the last couple of months That mainly happened because I was alone and not able to blog more often than once in a quarter The same applies to our dev automation mailing list which mostly only received emails from Travis CI with testing results Given that the team has been increased to 4 people now beside me this is Maja Frydrychowicz Syd Polk and David Burns we want to be more open again and also trying to get more people involved into our projects To ensure that we do not make use of the wrong communication channels depending where most of our readers are I have setup a little survey It will only take you a minute to go through but it will help us a lot to know more about the preferences of our automation geeks So please take that little time and help us The survey can be found here and is open until end of November 2015 https www surveymonkey com r 528WYYJ Thank you a lot automation Mozilla QA survey testing Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and finally a massive loss of core members Which means from the former 6 people only myself are remaining Since end of February all 5 former team members from Softvision are no longer participating in any of the maintained projects Thanks to all of them for the great help over all the last months and years But every project we own is now on my own shoulders And this is kinda hell of work with downsides like not being able to do as many reviews as I want for side projects One positive thing at least was that I got pulled back into the A Team at the same time With that move I m once more closer again to all the people who care about the basics of all test infrastructure at Mozilla I feel back home So what have I done the whole last quarter First it was always the ongoing daily work for maintaining our Mozmill CI system This was usually a job for a dedicated person all the last months The amount of work can sometimes eat up a whole day Especially if several regressions have been found or incompatible changes in Firefox have been landed Seeing my deliverables for Q1 it was clear that we have to cut down the time to spent on those failures As result we started to partially skip tests which were failing There was no time to get any of those fixed Happily the latest version of Mozmill is still working kinda nicely so no other work had to be dedicated for this project Most of my time during the last quarter I actually had to spent on Marionette especially building up wrapper scripts for being able to use Marionette as test framework for Firefox Desktop This was a kinda large change for us but totally important in terms of maintenance burden and sustainability The code base of Mozmill is kinda outdated and features like Electrolysis e10s will totally break it Given that a rewrite of the test framework is too cost intensive the decision has been made to transition our Mozmill tests over to Marionette Side effect was that a lot of missing features had to be implemented in Marionette to bring it at a level as what Mozmill offers Thanks for the amazing work goes to Andrew Halberstadt David Burns Jonathan Griffin and especially Chris Manchester For the new UI driven tests for Firefox Desktop we created the firefox ui tests repository at Github We decided on that name to make it clear to which product the tests belong to and also to get rid of any relationship to the underling test framework name This repository contains the harness extensions around Marionette a separate puppeteer library for back end and UI modules and last but not least the tests themselves As goal for Q1 we had to get the basics working including the full set of remote security tests and most important the update tests A lot of help on the security tests we got from Barbara Miller our intern from December to March She did great amount of work here and also assisted other community members in getting their code done Finally we got all the security tests converted My own focus beside the harness pieces were the update tests Given the complete refactoring of those Mozmill tests we were able to easily port them over to Marionette We tried to keep the class structure as is and only did enhancements where necessary Here Bob Silverberg helped with two big chunks of work which I m gladly thankful about Thanks a lot With all modules in place I finally converted the update tests and got them running for each version of Firefox down to 38 0 which will be the next ESR release and kinda important to be tested with Marionette For stability and ease of contribution we added support for Travis CI to our new repository It helps us a lot with reviews of patches from community members and they also can see immediately if changes they have done are working as expected The next big chunk of work will be to get those tests running in Mozmill CI to be renamed and the test reporting to use Treeherder Also we want to get our update tests for Firefox releases executed by the RelEng system to further reduce the amount of time for signoffs from QE About this work I will talk more in my next blog post So please stay tuned Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agendas the video recordings and notes from the Firefox Automation meetings Please note that since end of February we no longer host a meeting due to the low attendance and other meetings like the A team ones where I have to report my status automation continuous integration firefox marionette mozmill mozqa QA testing Mozilla Firefox Automation report week 51 52 2014 March 2 2015 Henrik Skupin 3 Comments In this post you can find an overview about the work happened in the Firefox Automation team during week 51 and 52 of 2014 I m sorry for this very late post but changes to our team which I will get to in my next upcoming post caught me up with lots of more work and didn t give me the time for writing status reports Highlights Henrik started work towards a Mozmill 2 1 release Therefore he had to upgrade a couple of mozbase packages first to get latest Mozmill code on master working again Once done the patch for handling parent sections in manifest files finally landed which was originally written by Andrei Eftimie and was sitting around for a while That addition allows us to use mozhttpd for serving test data via a local HTTP server Last but not least another important feature went in which let us better handle application disconnects There are still some more bugs to fix before we can actually release version 2 1 of Mozmill Given that we only have the capacity to fix the most important issues for the Mozmill test framework Henrik started to mass close existing bugs for Mozmill So only a hand full of bugs will remain open If there is something important you want to see fixed we would encourage you to start working on the appropriate bug For Mozmill CI we got the new Ubuntu 14 10 boxes up and running in our staging environment Once we can be sure they are stable enough they will also be enabled in production Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 51 and week 52 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meeting of week 51 and week 52 automation ci firefox marionette mozmill mozqa testing Mozilla Firefox Automation report week 49 50 2014 January 22 2015 Henrik Skupin 5 Comments In this post you can find an overview about the work happened in the Firefox Automation team during week 49 and 50 Highlights During the first week of December the all hands work week happened in Portland Those were some great and inspiring days full of talks discussions and conversations about various things Given that I do not see my colleagues that often in real life I have taken this opportunity to talk to everyone who is partly or fully involved in projects of our automation team There are various big goals in front of us so clearing questions and finding the next steps to tackle ongoing problems was really important Finally we came out with a long list of todo items and more clarity about so far unclear tasks In week 50 we got some updates landed for Mozmill CI Given a regression from the blacklist landing our l10n tests haven t been executed for any locale of the Firefox Developer Edition Since the fix landed we have seen problems with access keys in nearly each locale for a new test which covers the context menu of web content Also we would like to welcome Barbara Miller in our team She joined us as an intern via the FOSS outreach program as driven by Gnome She will be with us until March and will mainly work on testdaybot and the conversion of Mozmill tests to Marionette The latter project is called m21s and details can be found on its project page Soon I will post more details about it Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 49 and week 50 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meeting of week 48 Due to the Mozilla all hands workweek there was no meeting in week 49 automation continuous integration marionette mozmill mozqa testing Mozilla Firefox Automation report week 43 44 2014 December 18 2014 Henrik Skupin Leave a comment In this post you can find an overview about the work happened in the Firefox Automation team during week 43 and 44 Highlights In preparation for the QA wide demonstration of Mozmill CI Henrik reorganized our documentation to allow everyone a simple local setup of the tool Along that we did the remaining deployment of latest code to our production instance Henrik also worked on the upgrade of Jenkins to latest LTS version 1 565 3 and we were able to push this upgrade to our staging instance for observation Further he got the Pulse Guardian support implemented Mozmill 2 0 9 and Mozmill Automation 2 0 9 have been released and if you are curious what is included you want to check this post One of our major goals over the next 2 quarters is to replace Mozmill as test framework for our functional tests for Firefox with Marionette Together with the A Team Henrik got started on the initial work which is currently covered in the firefox greenlight tests repository More to come later Beside all that work we have to say good bye to one of our SoftVision team members October the 29th was the last day for Daniel on the project So thank s for all your work Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 43 and week 44 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meetings of week 43 and week 44 automation ci firefox jenkins marionette mozmill testing Mozilla Firefox Automation report week 41 42 2014 December 17 2014 Henrik Skupin Leave a comment In this post you can find an overview about the work happened in the Firefox Automation team during week 41 and 42 With the beginning of October we also have some minor changes in responsibilities of tasks While our team members from SoftVision mainly care about any kind of Mozmill tests related requests and related CI failures Henrik is doing all the rest including the framework and the maintenance of Mozmill CI Highlights With the support for all locales testing in Mozmill CI for any Firefox beta and final release Andreea finished her blacklist patch With that we can easily mark locales not to be tested and get rid of the long white list entries We spun up our first OS X 10 10 machine in our staging environment of Mozmill CI for testing the new OS version We hit a couple of issues especially some incompatibilities with mozrunner which need to be fixed first before we can get started in running our tests on 10 10 In the second week of October Teodor Druta joined the Softvision team and he will assist all the others with working on Mozmill tests But we also had to fight a lot with Flash crashes on our testing machines So we have seen about 23 crashes on Windows machines per day And that all with the regular release version of Flash which we re installed because of a crash we have seen before was fixed But the healthy period did resist long and we had to revert back to the debug version without the protect mode Lets see for how long we have to keep the debug version active Individual Updates For more granular updates of each individual team member please visit our weekly team etherpad for week 41 and week 42 Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agenda the video recording and notes from the Firefox Automation meetings of week 41 and week 42 automation ci firefox flash mozmill QA testing Mozilla Firefox Automation report week 39 40 2014 November 25 2014 Henrik Skupin Leave a comment In this post you can find an overview about the work happened in the Firefox Automation team during week 39 and 40 Highlights One of our goals for last quarter was to get locale testing enabled in Mozmill CI for each and every supported locale of Firefox beta and release builds So Cosmin investigated the timing and other possible side effects which could happen when you test about

    Original URL path: http://www.hskupin.info/tag/testing/ (2016-05-01)
    Open archived version from archive

  • Firefox Desktop automation goals Q1 2016 | hskupin.info
    be done for Linux right now it shouldn t matter that much given that nothing in our firefox puppeteer package is platform dependent so far Expanding testing to other platforms should be trivial later on For now the primary goal is to see test results of our tests in Treeherder and letting developers know what needs to be changed if e g UI changes are causing a regression for us If you are interested in more details have a look at bug 1237550 Documentation of firefox ui tests and mozmill ci We are submitting our test results to Treeherder for a while and are pretty stable But the jobs are still listed as Tier 3 and are not taking care of by sheriffs To reach the Tier 2 level we definitely need proper documentation for our firefox ui tests and especially mozmill ci In case of test failures or build bustage the sheriffs have to know what s necessary to do Now that the dust caused by all the refactoring and moving the firefox ui tests to hg mozilla org settles a bit we want to start to work more with contributors again To allow an easy contribution I will create various project documentation which will show how to get started and how to submit patches Ultimately I want to see a quarter of contribution project for our firefox ui tests around mid this year Lets see how this goes More details about that can be found on bug 1237552 automation ci continuous integration firefox marionette QA Post navigation Previous Post Review of automation work Q4 2015 Next Post Review of Firefox desktop automation work Q1 2016 Logging In Profile cancel Sign in with Twitter Sign in with Facebook or Comment Name Email Not published Website Notify me of follow

    Original URL path: http://www.hskupin.info/2016/02/02/firefox-desktop-automation-goals-q1-2016/ (2016-05-01)
    Open archived version from archive

  • continuous integration | hskupin.info
    fine and finally the entry scripts have been written Something special for them is that they even have different customers so extra configuration files had to be placed In detail it s us who run the tests in Jenkins for nightly builds and partly for release builds On the other side Release Engineering want to run our update tests on their own hardware when releases have to be tested By the end of September all work has been finished If you are interested in more details feel free to check the tracking bug 1192369 Mozmill CI Our Jenkins instance got lots of updates for various new features and necessary changes All in all I pushed 27 commits which affected 53 files Here a list of the major changes Refactoring of the test jobs has been started so that those can be used for mozharness driven firefox ui tests later in Q4 The work has not been finished and will be continued in Q4 Especially the refactoring for report submission to Treeherder even for aborted builds will be a large change A lot of time had to be spent in fixing the update tests for all the changes which were coming in with the Funsize project of Release Engineering Due to missing properties in the Mozilla Pulse messages update tests could no longer be triggered for nightly builds Therefore the handling of Pulse messages has been completely rewritten to allow the handling of similar Pulse messages as sent out from TaskCluster That work was actually not planned and has been stolen me quite some time from other projects A separation of functional and remote tests didn t make that much sense Especially because both types are actually functional tests As result they have been merged together into the functional tests You can still run remote tests only by using tag remote similar for tests with local testcases by using tag local We stopped running tests for mozilla esr31 builds due to Firefox ESR31 is no longer supported To lower the amount of machines we have to maintain and to getting closer what s being run on Buildbot we stopped running tests on Ubuntu 14 10 Means we only run on Ubuntu LTS releases from now on Also we stopped tests for OS X 10 8 The nodes will be re used for OS X 10 11 once released We experienced Java crashes due to low memory conditions of our Jenkins production master again This was kinda critical because the server is not getting restarted automatically After some investigation I assumed that the problem is due to the 32bit architecture of the VM Given that it has 8GB of memory a 64bit version of Ubuntu should have been better used So we replaced the machine and so far everything looks fine Totally surprising we had to release once more a bugfix release of Mozmill This time the framework didn t work at all due to the enforcement of add on signing So Mozmill 2 0 10 2 has been released automation continuous integration firefox jenkins marionette mozmill Software Mozilla Firefox Automation report Q2 2015 August 11 2015 Henrik Skupin 5 Comments It s been a while since I wrote my last Firefox automation report so lets do another one to wrap up everything happened in Q2 this year As you may remember from my last report the team has been cut down to only myself and beside that I was away the whole April Means only 2 months worth of work will be covered this time In general it was a tough quarter for me Working alone and having to maintain all of our infrastructure and keeping both of our tests Mozmill and Marionette green was more work as expected But it still gave me enough time to finish my two deliverables Ensure better visibility of Firefox UI test results for sheriffs and developers by uploading them to treeherder Finalize features for Firefox UI update tests and help to get them running on RelEng hardware Firefox UI Test Results on Treeherder With the transition of Mozmill tests to Marionette we no longer needed the mozmill dashboard Especially not since nearly no job in our Mozmill CI is running Mozmill anymore Given that we also weren t happy with the dashboard solution and the usage of couchdb under the hood I was happy that a great replacement exist and our Marionette tests can make use of Treeherder is the new reporting system for tests being run in buildbot continuous integration Because of that it covers all products and their appropriate supported branches That s why it it s kinda perfect for us Before I was able to get started a bit of investigation had to be done and I also checked some other projects which make use of treeherder reporting That information was kinda helpful even with lots of undocumented code Given that our tests are located outside of the development tree in its own Github repository some integration can be handled more loosely compared to in tree tests With that respect we do not appear as Tier 1 or Tier 2 but results are listed under the Tier 3 level which is hidden by default But that s fine given that our goal was to bring up reports to treeherder first Going into details will be a follow up task For reporting results to Treeherder the Python package treeherder client can be used It s a collection of different classes which help with authentication collecting job details and finally uploading the data to the server It s documentation can be found on readthedocs and meanwhile contains a good number of example code which makes it easier to get your code implemented Before you can actually upload reports the names and symbols of the groups and jobs have to be defined For our tests I have chosen the three group symbols Fu Ff and Fr Each of those stays for F irefox UI Tests and the first letter of the testrun name We currently have updates functional and remote As job name the locale of Firefox will be used That results in an output like the following Whether jobs are passing or failing it is recommended to always add the log files as generated by running the tests to the report It will not happen via data URLs but as artifacts with a link to an upload location In our case we make use of Amazon S3 as storage service With all the pieces implemented we can now report to treeherder and covering Nightly Aurora Developer Edition Beta and Release builds As of now reporting only happens against the staging instance of Treeherder but soon we will be able to report to production If you want to have a sneak peak how it works just follow this link for Nightly builds More details about Treeherder reporting I will do later this quarter when the next pieces have been implemented Firefox UI Update Tests under RelEng My second deliverable was to assist Armen Zambrano in getting the Firefox UI Update tests run for beta and release builds on Release Engineering infrastructure This was a kinda important goal for us given that until now it was a manually triggered process with lots of human errors on our own infrastructure That means lots of failures if you do not correctly setup the configuration for the tests and a slower processing of builds due to our limited available infrastructure So moving this out of our area made total sense Given that Armen had already done a fair amount of work when I came back from my PTO I majorly fixed issues for the tests and the libraries as pointed out by him All that allowed us to finally run our tests on Release Engineering infrastructure even with a couple of failures at the beginning for the first beta But those were smaller issues and got fixed quickly Since then we seem to have good results If you want to have a look in how that works you should check the Marionette update tests wiki page Sadly some of the requirements haven t been completely finished yet So the Quality Engineering team cannot stop running the tests themselves But that will happen once bug 1182796 has been fixed and deployed to production Oh and if you wonder where the results are located Those are not getting sent to Treeherder but to an internal mailing list as used for every other automation results Other Work Beside the deliverables I got some more work done Mainly for the firefox ui tests and mozmill ci While the test coverage has not really been increased I had a couple of regressions to fix as caused by changes in Firefox But we also landed some new features thankfully as contributed by community members Once all that was done and we agreed to have kinda stable tests new branches have been created in the repository That was necessary to add support for each supported version of Firefox down to ESR 38 0 and to be able to run the tests in our Mozmill CI via Jenkins More about that you will find below The only task I still haven t had time for yet was the creation of proper documentation about our tests I hope that I will find the time in Q3 Mozmill CI got the most changes in Q2 compared to all the former quarters This is related to the transition from Mozmill tests to Marionette tests More details why we got rid of Mozmill tests can be found in this post With that we decided to get rid of most of the tests and mainly start from scratch by only porting the security and update tests over to Marionette The complete replacement in Mozmill and all its jobs can be seen on issue 576 on Github In detail we have the following major changes Run all jobs with Marionette beside Firefox ESR 31 0 which is not supported by Marionette and ondemand update jobs because they still have to be run by Quality Engineering Reduced number of platforms We got rid of Windows Vista Ubuntu 14 10 and OS X 10 7 whereby the latter machines have been re used for OS X 10 10 No usage of a pre configured environments anymore but creating it from fresh for each test run by installing Python packages from the internal PyPI mirror Sending test results to treeherder and giving public access for everyone Stopped sending emails for failures to our mozmill ci mailing list in favor of having treeherder results All changes in Mozmill CI can be seen on Github Last but not least we also had two releases of mozdownload in Q2 Both had a good amount of features included For details you can check the changelog I hope that gave you a good quick read on the stuff I was working on last quarter Maybe in Q3 I will find the time to blog more often and in more detail Lets see automation ci continuous integration jenkins marionette mozmill Mozilla Firefox Automation report Q1 2015 May 12 2015 Henrik Skupin 2 Comments As you may have noticed I was not able to come up with status reports of the Firefox Automation team during the whole last quarter I feel sad about it but there was simply no time to keep up with those blog posts Even now I m not sure how often I will be able to blog So maybe I will aim to do it at least once a quarter or if possible once a month You may ask how it comes The answer is simple Our team faced some changes and finally a massive loss of core members Which means from the former 6 people only myself are remaining Since end of February all 5 former team members from Softvision are no longer participating in any of the maintained projects Thanks to all of them for the great help over all the last months and years But every project we own is now on my own shoulders And this is kinda hell of work with downsides like not being able to do as many reviews as I want for side projects One positive thing at least was that I got pulled back into the A Team at the same time With that move I m once more closer again to all the people who care about the basics of all test infrastructure at Mozilla I feel back home So what have I done the whole last quarter First it was always the ongoing daily work for maintaining our Mozmill CI system This was usually a job for a dedicated person all the last months The amount of work can sometimes eat up a whole day Especially if several regressions have been found or incompatible changes in Firefox have been landed Seeing my deliverables for Q1 it was clear that we have to cut down the time to spent on those failures As result we started to partially skip tests which were failing There was no time to get any of those fixed Happily the latest version of Mozmill is still working kinda nicely so no other work had to be dedicated for this project Most of my time during the last quarter I actually had to spent on Marionette especially building up wrapper scripts for being able to use Marionette as test framework for Firefox Desktop This was a kinda large change for us but totally important in terms of maintenance burden and sustainability The code base of Mozmill is kinda outdated and features like Electrolysis e10s will totally break it Given that a rewrite of the test framework is too cost intensive the decision has been made to transition our Mozmill tests over to Marionette Side effect was that a lot of missing features had to be implemented in Marionette to bring it at a level as what Mozmill offers Thanks for the amazing work goes to Andrew Halberstadt David Burns Jonathan Griffin and especially Chris Manchester For the new UI driven tests for Firefox Desktop we created the firefox ui tests repository at Github We decided on that name to make it clear to which product the tests belong to and also to get rid of any relationship to the underling test framework name This repository contains the harness extensions around Marionette a separate puppeteer library for back end and UI modules and last but not least the tests themselves As goal for Q1 we had to get the basics working including the full set of remote security tests and most important the update tests A lot of help on the security tests we got from Barbara Miller our intern from December to March She did great amount of work here and also assisted other community members in getting their code done Finally we got all the security tests converted My own focus beside the harness pieces were the update tests Given the complete refactoring of those Mozmill tests we were able to easily port them over to Marionette We tried to keep the class structure as is and only did enhancements where necessary Here Bob Silverberg helped with two big chunks of work which I m gladly thankful about Thanks a lot With all modules in place I finally converted the update tests and got them running for each version of Firefox down to 38 0 which will be the next ESR release and kinda important to be tested with Marionette For stability and ease of contribution we added support for Travis CI to our new repository It helps us a lot with reviews of patches from community members and they also can see immediately if changes they have done are working as expected The next big chunk of work will be to get those tests running in Mozmill CI to be renamed and the test reporting to use Treeherder Also we want to get our update tests for Firefox releases executed by the RelEng system to further reduce the amount of time for signoffs from QE About this work I will talk more in my next blog post So please stay tuned Meeting Details If you are interested in further details and discussions you might also want to have a look at the meeting agendas the video recordings and notes from the Firefox Automation meetings Please note that since end of February we no longer host a meeting due to the low attendance and other meetings like the A team ones where I have to report my status automation continuous integration firefox marionette mozmill mozqa QA testing Mozilla Firefox Automation report week 49 50 2014 January 22 2015 Henrik Skupin 5 Comments In this post you can find an overview about the work happened in the Firefox Automation team during week 49 and 50 Highlights During the first week of December the all hands work week happened in Portland Those were some great and inspiring days full of talks discussions and conversations about various things Given that I do not see my colleagues that often in real life I have taken this opportunity to talk to everyone who is partly or fully involved in projects of our automation team There are various big goals in front of us so clearing questions and finding the next steps to tackle ongoing problems was really important Finally we came out with a long list of todo items and more clarity about so far unclear tasks In week 50 we got some updates landed for Mozmill CI Given a regression from the blacklist landing our l10n tests haven t been executed for any locale of the Firefox Developer Edition Since the fix landed we have seen problems with access keys in nearly each locale for a new test which covers the context menu of web content Also

    Original URL path: http://www.hskupin.info/tag/continuous-integration/ (2016-05-01)
    Open archived version from archive

  • Misc | hskupin.info
    Engineer If you will take the advantage to work with engaged and talented people and to stay in touch with our world wide and powerful community go ahead and send your resume Mozilla Corporation is seeking a QA Engineer who is responsible for test execution of Firefox and Thunderbird products You will be developing feature test cases and run regression testing during releases You will also coordinate testing with the open source community and continually evangelize Firefox testing to the world Development of browsers is accelerating we re looking for engineers who can keep us ahead of the curve Responsibilities Create test plans and test cases from functional specifications Develop and execute test automation scripts Execute black box tests on a monthly release cycle Maintain test documentation and test cases Coordinate projects within the open source community Confirm create steps to reproduce identify regression ranges and clarify bug and feature requestion Monitor user feedback Requirements Strong knowledge of internet and browser technologies Strong understanding of test methodologies and test case development Strong problem solving and resolution skills Programming experience with in C C JavaScript Python Perl and XML Knowledge of Windows Mac Linux environments Experience with software testing lifecycle process release management and bug life cycle Strong verbal and written communication skills Flexibility in dynamic software environments BS or MS in Computer Science or related field Skills Desired Knowledge of web automation tools like Selenium Watir Silk Quicktest Pro and WinRunner Working knowledge of web security Experience with browser extensions and plugins Experience with virtual machines and multiple build environments Experience with testing open source products and open source community participation See also the full job description on jobvite com career job Mozilla Misc Welcome Dresden April 7 2009 Henrik Skupin 2 Comments Forgot to blog about last week So doing it right now It s a great feeling to stay in Dresden now Wonderful new flat which still needs a lot of work and new furniture but I feel home Misc Do I have an accent December 26 2008 Henrik Skupin Leave a comment Thanks to the posting from Fred I read about a quiz which tries to find the accent you are having Surprisingly I don t have an accent Should I be glad about What American accent do you have Your Result The Midland You have a Midland accent is just another way of saying you don t have an accent You probably are from the Midland Pennsylvania southern Ohio southern Indiana southern Illinois and Missouri but then for all we know you could be from Florida or Charleston or one of those big southern cities like Atlanta or Dallas You have a good voice for TV and radio Boston The Northeast The West The Inland North Philadelphia The South North Central What American accent do you have Quiz Created on GoToQuiz accent language quiz Misc Merry Christmas December 25 2008 Henrik Skupin Leave a comment I wish all my readers a merry Christmas Have a wonderful

    Original URL path: http://www.hskupin.info/category/misc/ (2016-05-01)
    Open archived version from archive