Automate what you see on a computer monitor
What is it? For what is it? Get it? Use it? Get help? Contribute?
Latest stable version: 2.0.5 —— Nightly builds: currently not available
A new webpage is on the way -- having a new SikuliX logo
... visit the news blog - enjoy!Sikuli is God's Eye
… in Huichol Indian culture: the power to see and understand things unknown.
(Thanks for the picture showing a Sikuli)
Sikuli was started somewhen in 2009 as an open-source research project at the User Interface Design Group at MIT by Tsung-Hsiang Chang and Tom Yeh.
Both left the project at Sikuli-X-1.0rc3 during 2012, when I decided to take over development and support and name it SikuliX.
What is SikuliX?
SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.
SikuliX supports as scripting languages
- Python language level 2.7 (supported by Jython)
- running RobotFramework text-scripts is supported (see docs)
- Ruby language level 1.9 and 2.0 (supported by JRuby)
- JavaScript (supported by the Java Scripting Engine)
Though SikuliX is currently not available on any mobile device, it can be used with the respective emulators on a desktop computer or based on VNC solutions. A solution for Android devices based on ADB (Android Debugging Bridge) is in an early experimental stage,
Besides locating images on a screen SikuliX can run the mouse and the keyboard to interact with the identified GUI elements. This is available for multi monitor environments and even for remote systems with some restrictions.
SikuliX comes with basic text recognition (OCR) and can be used to search text in images. This feature is powered by Tesseract.
SikuliX is a Java application, that works on Windows, macOS and most Linux/Unix systems.
— back to top —
When is SikuliX useful?
You want to automate some repetitive tasks in
- daily usage of applications or web pages
- playing games
- administration of IT systems and networks
You want to test applications or web pages, that are under development.
You want to create usage documentation or training material, that run live on the addressed application or web page.
You already have tools and workflows for that, but you want to add the specific features of SikuliX to enhance your approach and improve efficiency and outcome.
Using SikuliX means doing WYSIWYS: What You See Is What You Script ;-)
— back to top —
How can I get SikuliX?
Before thinking about using SikuliX, you have to assure, that you have a valid 64-Bit Java installation (version 8 or later).
But before downloading and fiddling around,
it is highly recommended to read about the gory details in QUICKSTART.
SikuliX can be downloaded from SikuliX's Launchpad page or
with a bit more information and advices from here
— back to top —
How do I use SikuliX?
In any case you should have read through QUICKSTART.
With SikuliX there are two rather distinct usage scenarios:
Scripting
One sets up and runs more or less complex SikuliX workflows based on searching images and acting on GUI elements with mouse and keyboard. For this one uses the so called SikuliX IDE, that brings some basic support for editing and running scripts.
In the Scripting usage scenario, you handle SikuliX scripts named you-name-it.sikuli, which are folders containing the script file and the images you need for the workflow. You either run the scripts from within the SikuliX IDE or from the command line.
To get on the road you should at least once scan the docs.
Programming
In a mature IDE (IntelliJ IDEA, Eclipse, NetBeans, ...) and/or some testing environment one integrates SikuliX as a feature library using the available API's while programming in Java or Java aware/based languages (Jython, JRuby, Scala, Clojure, ...).
If you work in this usage scenario, then you should at least once scan the docs and look out for special information on how to use with Java.
— back to top —
Where can I get help?
The first place to get help is to visit SikuliX's Launchpad page. You might ask a question in the Answers section or scan it for anything helpful. You might check the FAQ section. Even the Bugs section might shed light on your current problem.
… and even just asking Google, might reveal valuable information from elsewhere in the net.
More information on how to get help and insights.
— back to top —
How can I contribute?
The sources and hence the development base is on GitHub.Any contribution is welcome and highly appreciated.
Use any of the available channels, to get in contact.
— back to top —