NO.1 FOR RASPBERRY PI
• PYTHON MASTERCLASS • BUILD AN ONION PI • INTEL VS RASPI
www.linuxuser.co.uk
THE MONTHLY MAGAZINE FOR THE GNU GENERATION
HAS THE
RASPBERRY PI MET ITS MATCH? MinnowBoard: Review, Intel interview & spec face-off
Build extensions for XBMC Make
VS
Sync files with Unison
your own with Python
The quick & reliable way to save data
Studio Android Masterclass Google’s new IDE is here – get started today!
ISSUE 131
The other half of WordPress The story of co-creator Mark Little
Create a database with LibreOffice
ALSO INSIDE » LaTeX docs with LyX » Master the Vim editor » 14+ pages of reviews
Raspberry Pi Model B - 512MB with 8GB NOOBS SD Card
£32.99 Inc VAT
code SC 1
313058
£27.49 E x Vat
NOW IN STOCK! 5MP Raspberry Pi Camera £19.24 £16.03 Ex Vat
code SC1302358
visit cpc.co.uk for more information
C o vi nd si i t tio cp n c. s co ap .u ply k ,
cpc.co.uk
• Full range of project accessories • Kits and Cases • Expansion, Breakout and Prototyping Boards • Vast Range of Components • WiFi Dongles & HDMI to VGA Adaptors • Video and Audio Leads • Keyboards, Mice and Monitors • GPIO Ribbon Cables • Programming and User Guides
D F E R L E IV E E R Y
Get everything for your projects...
Welcome Your team of Linux experts…
to issue 131 of Linux User & Developer Mihalis Tsoukalos has over 15 years of
UNIX system administration and programming experience and has been using Linux since 1993. He is also proficient in Oracle database administration, Cisco IOS and Cocoa. In this issue Mihalis shows us how to sync files with Unison, a useful open source CLI tool (42-45).
Himanshu Arora is a software programmer,
open source enthusiast and Linux researcher. His articles have been featured on IBM developerWorks and Computerworld among others. He also blogs at mylinuxbook.com. This month he shows us how to make the most of Vim with his expert guide starting on page 62.
Michael Reed is a technology writer, and he’s
been hacking away at Linux for over 15 years. He specialises in desktop Linux solutions among other things. In issue 131 Michael shows us how to get started with databases using LibreOffice – it’s easier than you might think. See his work on pages 34-37.
Jon Masters is a Linux kernel hacker who has
been working on Linux for some 18 years, since he first attended university at the age of 13. Jon lives in Cambridge, Massachusetts, and works for a large enterprise Linux vendor. You can find his indispensable Kernel Column on pages 22-23 this month.
Gareth Halfacree is our new resident news
reporter and brings us the latest from all over the open source ecosystem, starting on page 14. Also this issue, Gareth speaks to Intel about its new dev platform MinnowBoard on page 6. You can also find his review of this potential RasPi-beater on pages 12-13.
Rob Zwetsloot studied aerospace engineering
at university, using Python to model complex simulations in class. Along with tutorials, reviews and more this issue, Rob tells us which IDE we should be using (pages 72-77) and walks us through the build process of an Onion Pi (pages 46-49).
Get Linux User for £4.19 » MinnowBoard exclusive per issue
This issue
» Build add-ons for XBMC Page 32 » Sync your files with Unison » Get started with Android Studio
Welcome to the latest edition of Linux User & Developer, the UK and America’s favourite open source and Linux magazine. When you’re trying to make the best Linux and open source read in the world, there’s nothing more useful than feedback from readers. There’s little point us being here if we don’t aim to give you a healthy dose of what you want, so we pay close attention to what you have to say. But what have you been saying? After collecting, formatting, graphing and a lot of stubble scratching, it’s clear many of you want more tutorials, bigger reviews and more introductory and intermediate developer content. As you’ll see this issue, we’ve dedicated more pages to tutorials and guides than ever before and we’ve even reformatted our distro reviews to make them more useful. We still need your help, though. If you’re keen to learn more about development in the open source field, we want to know what languages and frameworks you care about. We’ve been focusing very heavily on Python in recent months and intend to continue with this, but would you like to see Qt, Vala and C++ here too, or something else entirely? Let us know on Facebook and Twitter or simply drop me an email directly at
[email protected]. Russell Barnes, Editor
Get in touch with the team:
[email protected] Facebook:
Linux User & Developer
Twitter:
Buy online
@linuxusermag
Visit us online for more news, opinion, tutorials and reviews:
www.linuxuser.co.uk 3 Issue 131
NEW OPPORTUNITIES WITH NEW
DOMAINS Choose from over 700 new top-level domains! Create a short, memorable web address that perfectly fits your business or website theme, like fashion.blog, kitchens.online or john-smith.london. You can also make your website easier to find by getting new extensions for your current domain. With almost 20 million registered domains, 1&1 is Europe’s market leader for domain registration. Thanks to integrated forwarding features, domains registered with us can be quickly and easily connected with any website, no matter which web hosting provider you use. Find out more at 1and1.co.uk
DOMAINS | MAIL | WEB HOSTING | E-COMMERCE | SERVERS
NEW! PRE-ORDER
FREE
WITH NO OBLIGATION!*
1and1.co.uk
* Pre-ordering a domain name is not a guarantee that the domain will be registered. If your requested domain has already been pre-ordered, you will be added to a waiting list for that domain.
Exclusive
Intel’s big fish in a little pond
n Scott Garman is a Yocto Project engineer at Intel, and the company’s evangelist for the MinnowBoard
Intel’s big fish in a little pond
We chat to Scott Garman about Intel’s first foray into the world of Linux-powered open hardware development systems Scott Garman, the man Intel has appointed as the ‘evangelist’ for its first steps into open hardware development, has a history of embedded development. “I’m not a hardcore kernel hacker,” he tells us during an interview to commemorate the launch of the MinnowBoard single-board computer, “but a generalist who enjoys working with the big picture in mind.” A software engineer for the Yocto Project, a collaborative effort to make the development of Linux distributions for embedded platforms as simple as possible, working under Intel’s
6 www.linuxuser.co.uk
Open Source Technology Center, Garman has already given plenty back to the community. He’s responsible for Yocto-related training materials, including one of the most popular screencast introductions to the project, and is now leading the effort to introduce the first open hardware platform designed with Yocto firmly in mind: the Intel MinnowBoard. The MinnowBoard, reviewed on page 12, owes a clear debt of gratitude to devices like the Raspberry Pi and the BeagleBoard. Designed around Intel’s ‘Queens Bay’ platform, which has
at its heart a low-power 32-bit Atom processor, the board is designed as a development platform to bring the flexibility and familiarity of the x86 instruction set architecture to embedded systems – and to do so at a low cost. Most surprisingly of all, the device is open hardware: all specifications, schematics, board layouts and firmware packages are available for download and reuse under a permissive licence – a first for an Intel hardware product. Clearly, this is a departure from the norm, and one of which Garman is understandably proud.
Intel’s big fish in a little pond
How the MinnowBoard came about
InTervIew
The open hardware movement
n The MinnowBoard itself
is compact yet powerful, and boasts impressive I/O capabilities
“To my knowledge, MinnowBoard is the first hardware platform designed with the Yocto Project in mind” n Along with the power and reset buttons on the right, the MinnowBoard boasts four programmable buttons
How did the MinnowBoard come about? “I’d say development in earnest started happening at the end of 2012. MinnowBoard was an unusual project because of the way Intel and CircuitCo [the company behind the BeagleBoard and BeagleBone development platforms] collaborated on it. The Intel Yocto Project team provided input to help design the requirements for MinnowBoard around performance, openness, flexibility and standards. CircuitCo then used an Intel reference design – the ‘Queens Bay’ platform – and adapted it as needed. They also manufacture and sell the MinnowBoard, so it’s primarily their product, not Intel’s. The result is a win-win situation where our team got a great development platform and more, and CircuitCo has a compelling product platform to sell. “We’ve been really happy to work with CircuitCo on this, because they understand the open hardware/open community model so well, but we both had plenty of things to learn from each other during the process of bringing
n Open hardware – an offshoot of open source, where the schematics, component lists, and even the individual Gerber files for having printed circuit boards made are released under a permissive licence – is growing increasingly popular. Although the Raspberry Pi, one of the most popular low-cost embedded development platforms, is proprietary, many of its competitors are more open: as well as the MinnowBoard, the Gizmo, the BeagleBoard and its smaller BeagleBone offshoot, the Olimex OLinuXino family, and the microcontroller-based Arduino range are all open hardware. Open hardware can lead to rapid advances in features and technologies for the projects that adopt it, but as with open source software there are risks: the Arduino’s open nature has led to it being one of the most copied projects around, with Far Eastern factories churning out illegitimate clones by the dozen and misappropriating the project’s trademark in direct contravention of its otherwise extremely permissive licence.
up this first board. It’s turned out to be a great collaboration and we’re really pleased with the results. “In one sense, the Intel team working on this is extremely small. At the same time, I wouldn’t want to discount the contributions that many people have done to help make this happen, even if they weren’t involved with the project fulltime on a daily basis.” What led to the decision to choose the Queens Bay platform, with its ‘Tunnel Creek’ Atom processor, in the MinnowBoard? “Two main issues led us to use Tunnel Creek CPU for MinnowBoard. One, we were looking for longevity of the platform. Going with an open hardware model means that this is going to be an attractive board for embedded product companies to use and adapt for commercial products. I can tell you from experience, there is nothing more frustrating
n The MinnowBoard sits on four stilts to help aid the passive cooling of its 1GHz Intel Atom processor
7 www.linuxuser.co.uk
Exclusive
Intel’s big fish in a little pond
■ Larger and pricier than the Pi, the MinnowBoard offers full x86 compatibility
than developing a successful embedded product and then finding out that your components are about to enter end-of-life. Queens Bay is a platform used in IVI [In-VehicleInfotainment] devices, and was designed with a long lifespan – since people tend to own cars for a long time. It still has four to five years left in the product’s life cycle. “Two, the time to execute was now, so we weren’t about to wait for upcoming Intel platforms to roll out. Tunnel Creek met our main requirements and was available immediately, so we ran with it.” What advantages does the x86 architecture used in the Atom processor offer in the embedded space, compared to ARM or other RISC platforms? “Compatibility is likely the biggest advantage. Linux originated on the Intel 386, and whether you’re working on embedded or desktop applications, the toolchain, libraries etc were designed on and are pretty much guaranteed to work on x86. “Performance is another significant one. Countless person-years have been put into optimising compilers such as GCC to take full advantage of x86 platforms, so you can leverage that to its greatest benefit.” ■ The MinnowBoard includes eight GPIO pins and a dedicated expansion port for ‘Lures’
8 www.linuxuser.co.uk
Do you agree that the success of the ARMbased Raspberry Pi has demonstrated a clear demand for low-cost development boards from hobbyists? “Yes, absolutely. One of the things I appreciate and respect about the Raspberry Pi project is that it’s introducing so many new people to embedded Linux development. It used to be that embedded was a niche thing that software engineers specialised in, and now the embedded Linux community is becoming much more diverse, especially with younger people. I love that.” What does the MinnowBoard offer that other, more established development boards lack? “MinnowBoard stands out in its combination of performance, flexibility, openness and
standards. The Intel Atom CPU provides plenty of performance for high computational workloads. It also has strong I/O performance due to PCI Express powering its Gigabit Ethernet and SATA disk features. We also make PCI Express available through MinnowBoard’s expansion connector to enable high-speed I/O to custom hardware projects, for example interfacing with FPGAs or other hardware. “The MinnowBoard can be used for fun hobby projects, yet scales up to higher workloads. It’s expandable via several PC and embedded bus standards, and offers an environment for custom firmware development.
■ With Gigabit Ethernet and a real-time clock, it pulls ahead of the Raspberry Pi
Intel’s big fish in a little pond
How the MinnowBoard came about
INTERVIEW
“The open hardware model is very attractive in empowering your customers” MinnowBoard includes PC architecture standards including PCIe, USB and SATA, as well as embedded standards such as SPI, I2C, GPIO and even [a CAN bus] for automotive applications. It was designed with the Yocto Project in mind, which is an industry-wide standard for embedded Linux.” The MinnowBoard is open hardware – how important do you think that will be to its success in the market? “The open hardware model is very attractive in empowering your customers and allowing them to innovate in ways you can’t anticipate. Since people are free to reproduce and customise the MinnowBoard, the sky is the limit with this board’s potential. I think this will be fundamental to [its] success. “Keep in mind that open hardware is meaningful not only if the design files are available, but when all the parts are available through accessible distribution channels and the board’s price is not subsidised. If someone wants to rebuild it, or rebuild something similar, they can at about the cost it’s being sold for now. Not many boards can offer that potential.” What made Intel get involved in hobbyist-level embedded computing development? “We wanted MinnowBoard to become a useful platform for Yocto Project development that encouraged experimentation and the pursuit of fun projects in addition to more serious embedded applications, as a board we would use in Yocto Project training courses. And we needed it to be flexible enough so you could do interesting things in a classroom-style setting, which lines up with what hobbyists want, too.” With the board now in the hands of early adopters, have you seen any particularly innovative or exciting applications for the MinnowBoard emerge from the community?
“It’s still a bit early for me to have good visibility into this, as the board has only been available for about three weeks [at the time of the interview]. As I hear about community projects, I do intend to highlight them on our social media channels. “I am aware of someone who plans to use a MinnowBoard as part of a quality control system for 3D printers by making use of computer vision capabilities. I’ve also heard that folks in the FGPA design community are quite interested in the MinnowBoard due to its high-speed expansion capabilities. There’s also a group of students interested in building out learning exercises on the MinnowBoard, as a way to advance their embedded design skills.” Intel has something of a mixed history with the open source community - in particular in failing to release graphics drivers for selected Atom processor models. With the MinnowBoard being truly open, does this signal a shift in attitude towards the open source and open hardware communities at Intel? “The Open Source Technology Center at Intel is full of incredibly talented, well-known and passionate people who care deeply about openness. In recent years this group has grown dramatically in both size and influence, and it reflects the reality of the increased influence open source plays in the global software ecosystem. “Intel’s Core graphics have open source drivers that work with hardware acceleration, and the upcoming Bay Trail Atom platform makes use of it. This will address the issue you mentioned with graphics drivers on Atom. I’m looking forward to this, and I think Intel is definitely moving in the right direction in this regard.” Intel and CircuitCo have published a list of current and proposed add-on boards, dubbed Lures, for the MinnowBoard. Is there a particular add-on you would like to highlight? “One of the Lures that I’m sure will be quite popular is the Trainer Lure – the one based on an Arduino [microcontroller]. Combining the computational power of MinnowBoard with
Queens Bay
The code name for the combination of an Intel Atom E6xx-series processor with the EG20T controller hub (code-named Tunnel Creek and Topcliff respectively), Intel launched the Queens Bay platform in 2010 with the embedded market firmly in mind. Initially, the company targeted in-car computer systems for navigation and entertainment – so-called ‘infotainment’ devices – but claimed the platform would be equally at home in gaming, communications, point-of-sale, industrial and digital signage applications. The Tunnel Creek family of processors are not true system-on-chip designs, offloading much of the circuitry to the EG20T chip, but do include the CPU, a graphics processor supporting up to two simultaneous displays, an audio controller and a hardware video codec on a die some 46 per cent smaller than its predecessors. Until the launch of the MinnowBoard, however, using Queens Bay in hobbyist applications was almost impossible: Intel’s official development board was provided only to registered hardware partners, and came with the expectation that you would be placing an order for a few thousand processors once you had finished your design.
the embedded input/output capabilities and community of Arduino will open up a lot of interesting possibilities.” How critical is Linux and the Yocto Project to the MinnowBoard’s success? “We designed the board to be a Yocto Project development platform, and Yocto produces embedded Linux distros. So it’s pretty core to our purpose. It was the primary motivation behind the project itself – to my knowledge, MinnowBoard is the first hardware platform designed with the Yocto Project in mind. But most people using the MinnowBoard may have no knowledge of or interest in the Yocto Project – that’s okay, too.” Is the MinnowBoard likely to be the first in a family of open development boards from Intel, or is it merely an experiment for the company? “There are a lot of people within Intel who are excited about open hardware. I can’t speak [about] any specific future product plans, but I think the future is very, very bright.”
9 www.linuxuser.co.uk
Specifications
MinnowBoard vs Raspberry Pi
MinnowBoard Intel’s Atom-powered development board is the first to be Yocto Projectcompatible and runs Angström Linux. While considerably more expensive than the Raspberry Pi, it utilises the x86 architecture so offers computational power more akin to desktop and mobile computers. It’s also a very complete package, offering both Angström Linux on a microSD card and a properly rated power supply in the box. However, without decent driver support for the Intel GMA 600 GPU and a DVI-over-HDMI video interface (no HD audio or HDCP encryption), it lacks the same multimedia appeal as the Raspberry Pi.
■ SATA support brings mass storage within easy reach of MinnowBoard
■ The MinnowBoard supports tinkerers with two on-board LEDs and six buttons
■ Analogue audio input is a definite step up from the Raspberry Pi
■ Four stilts and bigger dimensions mean it towers over the RasPi
■ The stilts increase airflow to help keep the 1GHz Intel processor (under heat sink) cool
MinnowBoard Specs Price Dimensions Weight Operating system Processor Video Max resolution Memory
Output Input
Networking Real-time clock Extras
■ It has fewer GPIO pins, ■ Gigabit Ethernet means but includes two LEDs and faster networking – great four buttons to control if you’re planning a server
10 www.linuxuser.co.uk
■ MinnowBoard needs a staggering 2.5A to keep its lights flashing
Power
£162 106 x 115mm 119g Angström Linux (Yocto certified) supplied on microSD 32-bit Intel Atom E640T single-core at 1GHz Embedded Intel GMA 600 1366 x 768 (no HDCP encryption) 1GB DDR2 (shared), 4MB SPI flash (firmware storage) DVI-over-HDMI (no audio), analogue audio out Analog audio in, 2x USB, 1x SATA-2, 1x micro-USB, 1x mini-USB, 1x serial console, SD card slot, 1x micro-SDIO, 8xGPIO pins 10/100/1000 Ethernet Yes (no battery installed) Lure connector with CAN bus, HD audio, LVDS, IC, 3x PCI Express, SATA-2, SDIO, SPI bus, UART, 2x USB 5V at 2.5A (adaptor included)
VS
MinnowBoard vs Raspberry Pi
COMPARED
Raspberry Pi The Raspberry Pi has inspired thousands to try Linux for the first time and reinvigorated interest in bare-metal computing. It opens the door to the creation of fascinating Internet Of Things projects and drastically lowers the barrier of entry for both programming and physical computing. It’s not all plain sailing, though. Its low-price means the out of the box experience isn’t stellar. One still needs to acquire a microSD card, an HDMI cable and power supply, meaning ‘starter packs’ can retail upwards of £60. While its raw processing power is left in the shade by Intel’s x86 CPU, the Raspberry Pi continues to amaze users with its remarkable multimedia prowess.
■ Standard SD card support is good, but highspeed SD cards are worth the extra expense
Raspberry Pi Specs Price Dimensions Weight Operating system Processor Video Max resolution Memory Output Input Networking Real-time clock Extras
Power
£28 86 x 54mm 40g None included (Raspbian & others supported) ARM1176JZF-S (armv6k) single-core at 700 MHz VideoCore IV Full HD with HDCP encryption 512MB DDR2 (shared) DMI (with HDCP encryption), composite RCA, analogue audio out 2x USB, 1x micro-USB, SD card slot, 26x GPIO pins 10/100 Ethernet No (can be added via GPIO) DSI display connector, JTAG headers, CSI (camera module) connector, SPI bus, I2C, UART 5V at 700mA (not included)
■ A microSD card slot would have been preferable here
(Model B, Rev. 2)
■ The Pi’s small size makes it ideal for hiding behind a monitor or stashing in a project box
■ The RasPi could really do with a Lure port - even just two more USB ports would be great
■ The multimedia support is frankly stellar
■ 26 GPIO pins means there’s massive scope for tinkering. No analogue support, though, sadly
■ The Broadcom SoC is brilliant, even if it chugs occasionally
■ The audio on the RasPi is probably its weakest aspect. No mic support either
11 www.linuxuser.co.uk
Review
Intel MinnowBoard
■ The Lure connector, for addon boards, carries everything from CAN bus signals to three PCI Express lanes
■ An HDMI port offers DVI video connectivity, but does not carry digital audio
■ Two USB ports provide connectivity for external peripherals ■ Gigabit Ethernet gives the MinnowBoard powerful networking capabilities
DEV PLATFORM
Intel MinnowBoard
£162.83 ($200)
Intel’s answer to the Raspberry Pi is here – but at nearly six times the price, can it tempt buyers back to x86?
Pros
Powerful, flexible, open; a familiar x86 environment for newcomers to embedded development
Cons
Compared to ARM equivalents it’s expensive, and its performance lags behind AMD’s rival Gizmo device
Clearly, the MinnowBoard is no Raspberry Pi. Measuring 102mm on its shortest side, it’s significantly larger, requires a whopping 2.5A from a 5V power supply to run and costs almost six times as much as the credit-card-sized Pi. The MinnowBoard does offer something the Pi lacks, however: full x86 compatibility. At the heart of the MinnowBoard is one of Intel’s less powerful processors: the Atom E640T. Running at 1GHz, the single-core chip offers a 32-bit x86 implementation – already putting it on the back foot compared to the dual-core 64-bit APU found on rival AMD’s Gizmo, the closest device for comparison – while generating a surprisingly small
12 www.linuxuser.co.uk
amount of heat, allowing for passive cooling through a compact heat sink. With 1GB of RAM, the MinnowBoard offers a surprisingly powerful yet lightweight platform for embedded computing. Peaking, during testing, at 7W from the socket, it scored a respectable 95th percentile time of 11.49ms – almost five times faster than the Raspberry Pi at 51.45ms, but still some way behind the Gizmo’s impressive 9.87ms score. Add in the fact that the Gizmo can run two threads simultaneously and it’s clear Intel isn’t going to win on outright performance – although it has included Hyper-Threading support in the MinnowBoard for pseudo dual-core operation.
Intel MinnowBoard
x86-based embedded development board
REVIEW
■ A micro-SDIO slot plays host to a bundled SD card containing the Angström operating system ■ A small heat sink passively cools the Intel Atom E650T processor
■ Bundled risers lift the board up in order to allow the processor room to cool
Technical specs
Operating system Processor Video Memory Dimensions Weight Input/Output
Angström Linux (Yocto Certified) 32-bit Intel Atom E640T single-core at 1GHz Embedded Intel GMA 600 1GB DDR2, 4MB SPI Flash (Firmware Storage) 106 x 115mm 119g (excluding PSU) DVI-over-HDMI, Analogue audio in & out, 2x USB, 1x SATA-2, 1x USB device, 1x Serial console, Gigabit Ethernet, 1x micro-SDIO Extras Lure connector with CAN bus, HD audio, LVDS, I²C, 3x PCI Express, SATA-2, SDIO, SPI bus, 2x UART, 2x USB Power 5V at 2.5A
Treating the MinnowBoard as a standard computer, however, misses the point entirely. The device is designed for embedded development, and to help Intel regain a foothold in a market that has become almost completely dominated by ARM-based systems. Accordingly, it includes eight buffered general-purpose input-output pins, along with two user-controllable LEDs and a set of four switches. Together, these turn the MinnowBoard into a ready-to-run system for developing embedded applications – and unlike with ARM-based devices, those used to an x86 toolchain don’t have to learn anything new. The board also includes an expansion connector for addin daughterboards dubbed ‘Lures.’ Designed to be analogous to the ‘shields’ of the Arduino microcontroller, the port gives each Lure access to considerable potential: as well as three PCI Express lanes, an SDIO channel, two USB channels and more, the port carries everything from CAN to I²C buses. Although there are no Lures available on the open market at the time of writing, several are in development – including one which will offer compatibility with Arduino shields. That doesn’t mean the main board is bereft of connectivity: a micro-SDIO slot for the boot device, two USB ports, analogue audio connections, Gigabit Ethernet, and mini- and micro-USB ports for acting as a USB device or as a debug serial console are included. The board also features a SATA-2 port, offering up to 3Gbps of throughput to a mass storage device – with a second port available through the Lure expansion port if required.
Also consider
■ Four switches combine with two usercontrollable LEDs to get users started with the board’s capabilities
At present, the MinnowBoard ships with a bare-bones installation of Angstrom Linux – no friendly out-of-box experience here. With full Yocto Project certification, however, rolling your own OS isn’t a challenge and Intel is in talks with distributions including Ubuntu to add support for the board’s somewhat unique 32-bit UEFI firmware. Using documentation – still in progress – from the project’s founders, the MinnowBoard is quick to offer up its GPIO capabilities. The only real disappointment comes from the HDMI socket, which only carries a DVI signal and not the audio required of a full HDMI implementation. There’s also no support for HDCP encryption, although anyone considering using the MinnowBoard as a media playback system has probably missed the point of its design and features. Gareth Halfacree
Sage Gizmo
$199 (£154.29)
Based on an embedded AMD APU, it’s far more powerful than the MinnowBoard. Embedded graphics aid computer performance and a bundled expansion board makes getting started easy. It does, however, require more power under load. gizmosphere.org
Raspberry Pi
Summary
£28.07
The MinnowBoard is a welcome foray from Intel into the world of open hardware and offers considerable potential. While its performance may lag behind the rival Gizmo board, the Lure connector holds promise for some impressive add-ons to come – but its high price means the Raspberry Pi has little to fear from this upstart.
The Pi’s ARMv6 processor is considerably less capable than the MinnowBoard’s Atom – but at almost a sixth the price, it’s easy to overlook this in favour of having spare cash for add-on hardware and tinkering components. raspberrypi.org
More information minnowboard.org
13 www.linuxuser.co.uk
OpenSource
Your source of Linux news and views www.linuxuser.co.uk For the latest news and views
Email us directly…
[email protected]
OpEn SciEncE
Linux Foundation leads open science initiative Partnership with OpenBEL will make sharing scientific data easier A collaboration between the Linux Foundation and Selventa’s freshly opened OpenBEL project could help push the ideals of open source development further into the realms of scientific research, the groups have announced. Previously a closed-source proprietary platform, OpenBEL – the Open Biological Expression Language – was designed to help users to capture, store, share and use life sciences content through what its creators describe as a “knowledge engineering platform.” Addressing the difficulties with sharing and using data, the team behind OpenBEL has been using it for ten years – but the platform was only opened up in June last year, a move that the Linux Foundation is claiming will be the making of the project. “All of us are smarter collectively than any one of us is by ourselves, and Linux is one of the greatest examples of that principle,” claimed Jim Zemlin, executive director at The Linux Foundation. “We are able to take what we know about Linux and collaborative development and transfer that to new industries. OpenBEL represents an amazing opportunity for openness and collaboration to
nOpenBEL’s Ted Slater extols the virtues of Linux and collaborative development
advance science, and we’re happy to impart our knowledge of collaborative software development to leaders in the life sciences industry. Successful open source projects don’t just host code; they make use of a full suite of open source best practices to quickly gain adoption and collaboration. We aim to help OpenBEL achieve even more success.” “The Linux Foundation hosts the largest collaborative project in the history of computing: Linux,” stated Ted Slater, project
nOpenBEL provides a platform for the capture and collaborative analysis of life sciences data
14 www.linuxuser.co.uk
lead for OpenBEL. “It is the standard by which all open development projects measure themselves. We know our industry can learn a lot from this neutral steward of open development and governance… Also, by hosting OpenBEL at The Linux Foundation, we have access to a variety of important services to help facilitate collaborative development, allowing our teams to focus on our subject matter: life science.” Hosting the OpenBEL platform with the Linux Foundation, its creators hope, will boost its adoption. Since going open source in June 2012, the project has already been adopted by organisations as diverse as the Harvard Medical School, the University of California at San Diego, the Fraunhofer Institute, and pharmaceutical giants Novartis and Pfizer. The groups hope that wider adoption of OpenBEL will lead to a pooling of information on life sciences, with the platform allowing for easy dissemination and analysis using a standardised set of computable networks and application programming interfaces (APIs). The deal with OpenBEL is the latest of the Linux Foundation’s Collaborative Projects, which include partnerships with the Yocto Project, Xen Project, FOSS Bazaar, and mobile platforms MeeGo and Tizen. Details on the project, and links to the source code, are available on the official OpenBEL website at openbel.org.
“All of us are smarter collectively than any one of us is by ourselves” Ted Slater
News
ThelatestintheLinuxcommunity
OPEn SOURCE
nThe DevShare programme introduced by SourceForge owner Dice Holdings has drawn strong criticism
efforts; its critics claim it’s nothing more thanmalware. “SourceForge, once a mighty force for the good of open source, has fallen far from its previous lofty heights,” opined Red Hat developer Justin Cliff in a post to the Gluster Project blog. “I’m not against monetisation at all, we all have lives and need to pay our bills. Butnotthroughabusingusertrust.Notthrough preyingontheunskilledorunwary.Tomisquote MargeSimpson:‘Theynotonlycrossedtheline, theythrewuponit.’” SourceForge, now owned by Dice Holdings, has defended the initiative. “SourceForge will always respect the rights of our users and we will never infringe on them. DevShare offers a transparentinstallationflowthatgivesusersall the necessary information to make educated choicesaboutwhatsoftwaretoinstall.” The decision to launch the programme, currentlyinbeta,hasledtocallsforhigh-profile projects to move to alternative hosts; others – including FTP client FileZilla – have, however, alreadysigneduptoDevShare.
OPEn SOURCE
X.Org loses non-profit status The Foundation behind X.Org has lost its status as a 501(c)(3) non-profit group in the US, following a failure to file taxes with the US Internal Revenue Service for three consecutive years. “I was taken by surprise that the IRS hit ussorudely,”FoundationaccountantStuart Kreitmansaidatthecompany’smostrecent board of directors’ meeting. “I’ve had little issueswithmyownreturnsandhavealways foundthemtobereasonableandfriendly.” Kreitman has, however, admitted that the tax returns have not been filed, but argued that the organisation has “never filedreturns.”
nThe X.Org Foundation is considering letting others take care of its paperwork in future
TheFoundationisnowconsideringjoining an umbrella organisation in order to reduce thepaperworkrequired of its members and prevent this sort of embarrassing situation fromrecurring.
3 - 5 October 2013
Open source hosting outfit SourceForge has been criticised for bundling selected binary releases in advertising-laden packages. LaunchedinJulythisyear,DevSharebundles software with add-ons such as AnchorFree’s HotSpotShield. SourceForge claims it provides an easy way for developers to monetise their
Le Beffroi, Paris » » France » openworldforum.org The first European summit designed to bring together technical experts and political representatives, Open World Forum includes speakers ranging from Microsoft’s Frederic Aatz to the CIO of the French Culture and Communication Ministry.
19 - 20 October 2013
“They not only crossed the line, they threw up on it”
Open World Forum ’13
OggCamp
21 - 23 October 2013
SourceForge DevShare branded ‘malware’
Linux calendar
LinuxCon Europe
21 - 23 October 2013
OPEn SOURCE
CloudOpen Europe
» LJMU Art & Design Academy, Liverpool » England » oggcamp.org The largest open source and free culture event in the UK enters its fifth year. Based around the ‘unconference’ format, there will be three fluid tracks along with a series of scheduled speakers throughout the weekend.
» Edinburgh International Conference Centre » Scotland » events.linuxfoundation.org The largest event covering Linux in general – rather than specific distributions – comes to Scotland, with over a hundred planned sessions. Followed by the Linux Kernel Summit, Automotive Linux Summit and Embedded Linux ConferenceEurope,allinEdinburgh.
» Edinburgh International Conference Centre » Scotland » events.linuxfoundation.org The second annual CloudOpen conference looks to build on the success of the first, helping push forward the use of open source technologiesincloudcomputing.
15 www.linuxuser.co.uk
OpenSource
Your source of Linux news and views www.linuxuser.co.uk For the latest news and views
Email us directly…
[email protected]
GOVERNMENT
Valencia completes LibreOffice switch Local government coffers already €1.3 million richer
The Valencian regional government in Spain has completed its planned switch from proprietary software to LibreOffice. The project to migrate to an open source alternative began in 2012, and has already been credited with saving the government €1.3 million in software licensing fees. “Apart from economic benefits, the commitment to free and open source software brings other advantages, including having the solutions available in the Valencian language as well as in Spanish, and IT vendor independence, which encourages competition,” said Sofia Bellés, director general of the Valencian government ICT department. “We also have the freedom to modify and adapt the software to our every need.”
■ Moving to LibreOffice has saved the Valencian government a small fortune
Valencia has long been at the forefront of open source adoption, having pushed a programme of Linux adoption in schools back in 2005. Covering 110,000 PCs installed across all regional schools, the move to Lliurex Linux is claimed to have saved over €30 million so far. “Installation of the free office suite is part of the regional government’s strategic commitment to its use of free software. It will not only help save costs for licences,” Bellés
added, “but also boosts the development of the local ICT sector, promotes the use of Valencian in the digital world and improves interoperability and security of the administration’s IT systems.” The move has been lauded by the European Commission’s JoinUp programme, which looks to improve interoperability between public administrations through standardisation, suggesting other governments may follow suit.
HARDWARE
ARM begins Internet of Things push Aims for a trillion devices sold by 2025 Cambridge-based low-power processing giant ARM is throwing its weight behind the Internet of Things, forming a new IoT business unit with the aim of selling a trillion devices by 2025, and purchasing IoT specialist Sensinode Oy. Claimed by its proponents to be the next logical evolution of the internet, the Internet of Things looks to equip everything possible with internetconnected sensors and interactivity as the means to making people’s lives easier. ARM plans to integrate Sensinode’s 6LoWPAN and CoAP standards with its own Cortex-based mbed microcomputing platform, which it sells both for embedded hardware development and in breadboard-friendly format for hobbyist use. “Sensinode is a pioneer in software for lowcost low-power internet connected devices and
16 www.linuxuser.co.uk
has been a key contributor to open standards for IoT,” claimed ARM’s John Cornish of the deal. “By making Sensinode expertise and technology accessible to the ARM partnership and through the ARM mbed project, we will enable rapid deployment of thousands of new and innovative IoT applications.” The move comes as industry analysts predict a pending explosion in the number of connected devices, with Bill Morelli of IHS offering an estimate of 30 billion connected devices by 2020 – a figure that ARM is going to do its best to exceed. Small-scale IoT projects, including printers that act as Twitter gateways and environmental monitoring systems, often make use of ARMbased microcomputing systems – most
■ ARM is pushing its technologies, including the mbed prototyping platform, as the future of IoT
frequently, these days, the low-cost and highperformance Raspberry Pi or its more flexible alternative the BeagleBone Black – while others use lower-cost microcontroller devices from Atmel, Texas Instruments and others.
News
The latest in the Linux community
open source
linux
Torvalds celebrates linux 3.11 Following his decision to name Linux 3.11 ‘Linux for Workgroups,’ Linus Torvalds has posted a nostalgic message celebrating its final release candidate. “Hello everybody out there using Linux – I’m doing a (free) operating system (just a hobby, even if it’s big and professional) for 486+ AT clones and just about anything else out there under the sun,” Torvalds wrote. “This has been brewing since April 1991, and is still not ready.” The message is an amusing echo of his 1991 initial Linux release notice, which called for Minix users to try Linux and claimed it “won’t be big and professional like GNU.”
viDeoGames
metro: last light heads to linux
Linux gaming no longer being ignored by triple-A developers. Developer 4a Games has confirmed that firstperson shooter metro: last light is heading to linux following its debut on windows earlier this year. Due to arrive on Steam for Linux later this year, Metro: Last Light is one of the first big-name titles to be released on Linux in the same year as its Windows counterpart – and marks a growing trend for development houses to consider Linux as a valid target market for their software. Development of the Linux release was performed in-house at 4A Games, with the company crediting its custom game engine for making it easy to port to alternative platforms. The game will also be launching on OS X around the same time.
n4A Games has ported Metro: Last Light to Linux, for launch later this year
“We are very happy with the results,” said 4A’s CTO Oles Shishkovstov. “We hope that Mac and Linux gamers will appreciate our efforts to create the best possible version for their machines.”
malware
Hand of Thief Trojan targets linux
Experts warn users to beware as malware spreads a Trojan horse, dubbed the Hand of Thief, has been discovered targeting linux users and attempting to grab banking credentials and other sensitive data. The Trojan, discovered by RSA researchers, includes sophisticated anti-monitoring and anti-virtualisation techniques which make it hard to analyse. Sold for around $2,000 on underground sites, the Trojan is claimed to be effective against all common Linux desktop distributions. Anti-virus experts are positioning the Trojan as proof that such protective software is a requirement on Linux. “It’s yet another reason why Linux users shouldn’t be complacent about their computer security, and run an anti-virus program,” claimed former Sophos consultant Graham Cluley. “The statement that the Linux platform is absolutely secure now seems even more illusive,” added Avast!’s Peter Kálnai. However, the Trojan can only install when provided with root access – something the majority of modern desktop distributions shut off behind a password. By being careful about what is installed and not providing a password to an unexpected pop-up dialog, Linux users should largely be protected from its ravages. Those who frequently install software from outside their distribution’s official repositories, however, should take heed: as the popularity of Linux grows, sadly so too does its attractiveness to criminals.
Does your current database support provider guarantee a 15 minute response?
We do. 2ndQuadrant’s Platinum Production Support for PostgreSQL provides a guaranteed 15 minute (human) response, 4 hour workaround and guaranteed bug fix within 24 hours. 24 hours a day, 365 days a year. PostgreSQL is used worldwide for mission critical applications. Now you can choose this open source database, safe in the knowledge that if you ever need help, it‘s less than 15 minutes away. With guaranteed fix times and access to leading Postgres experts. 2ndquadrant.com/support
.
.
.
.
.
PostgreSQL Development 24x7 Support Training Services 2ndQuadrant Ltd. 7200 The Quorum, North Side, Oxford Business Park, Oxford, OX4 2JZ, UNITED KINGDOM www.2ndquadrant.com TEL +44-870-766-7756
Opinion
The free software column
open source
open source robotics
Linux is the platform for robotics
Linux is increasingly being used for cutting-edge robotics – opening up the field to anyone interested in learning more
Alan broun is currently studying
for a PhD at the Bristol Robotics Laboratory. He is also the managing director of Dawn Robotics Ltd – www.dawnrobotics.co.uk
robotics is a fascinating subject. It links computers to the physical world, allowing them to move around, sense their environment, and to interact with it. Building your own robot, however, has traditionally required that you spend a huge amount of time and energy to get even the most basic of robotic systems up and running. You need mechanical engineering skills to build the chassis or body, electronic engineering skills to wire up the motors and the sensors, and programming skills to animate the robot, to control its body to get it to do what you want. These barriers to entry mean that people entering into the field of robotics – either in search of a stimulating pastime, or perhaps as part of more formal research – can quickly be overwhelmed by the sheer amount of work required. The initial dreams of building a robot to fetch your beer or to walk your dog get pushed further and further back, as more and more time is sucked up just getting the basics working. Things are changing, however, and the barriers to entry are falling. The relentless march of progress in consumer goods means that a lot
18 www.linuxuser.co.uk
of commonly used robotics hardware – such as cameras, accelerometers and communication devices – are now both much cheaper and much smaller than they were ten years ago. Actually making use of electronic sensors, and motors, is also a lot easier now, as platforms like the Arduino make connecting electronic components to your computer, and controlling them, both quick and simple. On top of all this, small, powerful computers such as the Raspberry Pi and the BeagleBoard, to name but two, mean that you can now cram much more processing power into your robots at an affordable price. In order to put this hardware to good use, however, you need software, and as luck would have it, Linux is the platform where the most exciting developments in robotic software are taking place. Writing the software for modern robots can be one of the most involved and complicated parts of the process. Software is needed for controlling motors, reading values from sensors and, possibly most importantly, to provide high-level control and AI. A number of distributed software environments have been produced to try to ease the development of robotic software. But the Robot Operating System (ROS) produced by a company called Willow Garage in Silicon Valley is arguably one of the most successful. ROS is not actually an operating system, but rather a BSD-licensed open source software framework which runs on Linux. It allows interfaces to be defined for common bits of robotic software, such as the drivers for cameras and motors, and it allows this software to be run as a large number of separate processes called ‘nodes’ – either all on one machine, or transparently distributed over a network of machines.
Over the last five years, ROS has dramatically eased the process of writing robotic software. Now you can download a large number of precompiled packages to quickly allow you to hook up common sensors such as cameras or Microsoft’s Kinect. High-level services such as inverse kinematics, map building and speech recognition are easy to plug in, and ROS also provides a great selection of visualisation tools so that you can see what’s going on from your robot’s point of view. When you need to write your own software, you can do it in the language you choose. Low-level motor drivers can be written in C and then communicate over ROS’s network layer with high-level control processes, written in a language such as Python or Java. Not having access to robotic hardware is no bar to entry, as ROS also provides support for simulators such as Gazebo. ROS has been embraced by the robotics research community and so lots of simulations of cuttingedge robots are now provided by the teams who built them. So, for absolutely no cost at all, you can get access to Willow Garage’s PR2 robot, NASA’s Robonaut and Boston Dynamics’ Atlas robot. This last robot is currently being used in the DARPA Robotics Challenge, which seeks to get a humanoid robot to drive a car and move around a disaster area, so it’s ambitious stuff. Now is an exciting time in robotics, and Linux is the platform on which some of the coolest stuff is happening. If you haven’t tried it yet, then I would urge you to download and have a play with ROS, start building yourself a robot and join in all the fun!
“Not having access to robotic hardware is no bar to entry”
Can you volunteer for Code Club? Code Club is a nationwide network of volunteer-led after school coding clubs for children aged 9-11. We need people who know how to program computers to volunteer to run a club at their local primary school, library or community centre for an hour a week. We create the projects for our volunteers to teach, the projects we make teach children how to program by showing them how to make computer games, animations and websites. Get involved, let’s teach the next generation to code!
Visit www.codeclub.org.uk to find out more
OpenSource
Your source of Linux news and views
the Open SOurce cOLumn
Fair game
Simon fears the future of gaming is the future of technology
Simon Brew is a technology writer and editor, working across the Linux, Windows and Mac OS X platforms
I play games. I’ve always liked playing games, and suspect I’ll continually sneak one or two in for the rest of my days. If you talk to my editor, he’ll obviously tell you that I’m the most reliable hitter of a deadline that he’s ever met and that my words arrive perfectly formed, requiring the barest of edits before they’re laid before your
eyes. Scratch that: thanks to computer games, I’ve slipped under the proverbial door seconds before it slams shut more times than I’d care to admit. But games are changing. For someone who was brought up with the spirit of 8-bit computing, where anyone could spend a day or two locked up in their bedroom coding a game, the current climate is all a little disconcerting. Take Watch Dogs: this is a big game that a big publisher is making, that before it’s even been released has been put on the fast track for a bigscreen movie. Or the evil of Candy Crush Saga, the mobile game revolution that doesn’t want any money off you up front, but is happy to take as much off you as it can once it’s got you cosy and sitting on the sofa. Games are interesting, and always have been, because they’re at the forefront of control. I think back to sticking dongles in the back of my Commodore Amiga to stop people pirating a game (naturally, the only people inconvenienced turned out to be the people who legitimately coughed up), or lining up pictures on wheels. Then, as technology evolved, so did the methodologies of control. PC gaming, through
n King’s Candy Crush Saga – a ruthless money-making machine
20 www.linuxuser.co.uk
Windows, has taken things to scary levels. At one stage, the publisher Ubisoft implemented a system whereby you had to go online and prove you were legitimate once a day, else you weren’t allowed to play the title that you’d legally bought. Naturally, one day its servers weren’t working properly and people weren’t allowed to play their game. Those running pirates copies could. Microsoft has tried to implement a similar system with its upcoming Xbox One games console, although it’s had to perform a significant U-turn when the volcano of public opinion erupted. Still, those buying a game for an Xbox One machine will need to ‘activate’ it. A friendly word that, isn’t it? It sounds so reasonable and happy. Just do one thing and everything will be active and working. What could be easier? The truth, of course, is that it’s a further eradication of users’ rights in the proprietary software sector. The problem is that what happens in videogaming increasingly becomes a dry-run for what we get in the broader technology ecosystem. Publishers know that people will endure a degree of hassle to play a game, and they take advantage of it ruthlessly. Now, we’re at a point where Adobe’s Creative Suite of tools won’t even be sold in a box any more – it’s the cloud version or bust, and a subscription. Someone’s been looking at the World Of Warcraft model. Even in the app sphere, where the spirit of the bedroom programmer should still be alive and well, there’s a corporate sheen that’s taking the fun off things. That Candy Crush Saga model has proven to be the proverbial grail for many, in that the appearance of a free, open approach is just a disguise for one of the most ruthless gaming money-making machines I’ve seen in recent times. It’s why the good ones need supporting, of course. But for all the marches of progress, the loss of an ethos has become real collateral damage.
Opinion
The free software column
open source
the Free soFtware column
a Gpl breakthrough
Samsung is releasing the code for its exFAT file system driver for Linux under the terms of the GPL. This is an important breakthrough because of the participation of both Samsung and the GPL Compliance Project for Linux Developers
richard hillesley writes about
art, music, digital rights, Linux and free software for a variety of publications
non-profit Foss organisation the software Freedom conservancy has reached an amicable agreement with samsung to release the code for samsung’s exFat file system driver for linux under the terms of the Gpl. The exFAT driver code came to light through its inadvertent release via GitHub (lwn.net/Articles/560424/) and the use of a binary version in a Samsung Linux-based tablet. The Software Freedom Conservancy, led by Bradley Kuhn, works with developers and manufacturers to ensure compliance with the GPL. The greater part of the job is to point out the legal obligation to manufacturers and to help them achieve compliance, and most will happily comply.
As we all know, the GPL makes a simple pact between the coder and the user. Anyone can take, modify, copy, share and redistribute the software and the code, but must pass on the same rights to subsequent users of the software, including any modifications to the code. The coder, who is usually (but not always) the copyright holder, gains because enhancements to the code are fed back through the development process. The manufacturer gains because thirdparty developers become involved who may bring new dimensions to the code. The user gains because the code remains free and the obligation is mutual, meaning that every other user has the same obligation to feed their changes back. The only requirement of the GPL is that the source code be made accessible to end users, but this requirement is often ignored, especially when the code is reused in firmware and embedded devices. GPL code is used in thousands of devices, but many don’t comply with the terms of the licence. The role of Software Freedom Conservancy is to alert manufacturers to failures in compliance and to help them to reach an amicable resolution. Most violations are resolved without court proceedings. In very rare cases this may involve litigation but as Jeremy Allison, a board member of Conservancy, observes: “The point is not to punish people for making mistakes, but to bring them into compliance. When people get into trouble it’s usually down to laziness and inconvenience. It’s usually a case of ‘I can’t be arsed, and it’s too much effort to do it right, so I’ll just use it’.”
“Sharing the code is useful to everyone, but the terms of the GPL haven’t always been enforced”
The great majority of infringements are not deliberate, and can be attributed to misunderstandings and lack of attention to detail. Manufacturers of mobile devices operate in a rapidly changing environment with short product cycles and shorter time-to-market. The market for firmware and mobile devices is highly competitive, and every new product comes to market with a new range of features. Failure to comply with the GPL is usually inadvertent, but releasing the source code is a small price to pay when set aside the considerable advantages of cost-effectiveness, speed to market, and the accessibility of pre-written and tested code that free software offers, especially when it is remembered that it is only the GPL’d code that has to be made available to others. Sharing the code is useful to everyone, but the terms of the GPL haven’t always been enforced because the coders have other things to do, or the copyright has been assigned to corporate entities who don’t care about the licence beyond their immediate needs. For this reason it is an important development that significant contributors to the Linux kernel, in the shape of Conservancy’s GPL Compliance Project for Linux Developers, have become involved in helping to ensure compliance with the terms of the licence, and that Samsung, a major manufacturer, was not only a willing and amicable partner in releasing the code, but was happy ‘to talk publicly about the matter’, which may make it easier to achieve compliance from other manufacturers in the future. The drawback, in this instance, is that exFAT is owned by Microsoft, is a proprietary file format and has patent issues. The exFAT driver code can only be deployed by manufacturers or distributions that have obtained a licence from Microsoft.
21 www.linuxuser.co.uk
OpenSource
Your source of Linux news and views
JOn MaSterS
the kernel column The latest developments in the kernel community, including changes to Linux 3.11
Jon Masters is a Linux kernel hacker who hasbeen working on Linux for some 18 years, since he first attended university at the age of 13. Jon lives in Cambridge, Massachusetts, and works for a large enterprise Linux vendor. He publishes a daily Linux kernel mailing list summary at kernelpodcast.org
Linus torvalds announced the latest release candidate of the 3.11 ‘Linux for Workgroups’ kernel on the 22nd anniversary of his original ‘Hello everybody’ email first introducing Linux back in 1991. The last few weeks have been largely quiet in terms of changes merged into the mainline kernel, typical both of late summer and of the late stages in a kernel release cycle. It would seem that temporary fixes for the ongoing Windows 8 ACPI compatibility issues – which cause display backlights (now managed directly by certain Windows graphics drivers, rather than in the ACPI firmware) to misbehave on some laptops – will suffice for this release, but will be
22 www.linuxuser.co.uk
raised again in the next cycle. If all goes according to plan, the final release should be out very shortly, in time for a summary in next month’s issue of Linux User & Developer. Every Linux release has a (code) name, typically something very silly, and usually whatever Linus dreams up at the time he opens the merge window (the period of time during which disruptive kernel changes are allowed) for a new kernel release. But from time to time, a release name has a deeper meaning. The 3.11 release came close to happening on the 20th anniversary of the original Windows 3.11 ‘Windows for Workgroups’ release, but it was not to be. Still, Linus had some fun with the celebration of the 22nd anniversary of his original Linux announcement, posting a Google+ post in which he parodied himself, saying: “I’m doing a (free) operating system (just a hobby, even if it’s big and professional) for 486+ AT clones and just about anything else out there under the sun. This has been brewing since 1991, and is still not ready.”
nUMa development
Linux Weekly News recently noted that there can often seem to be themes to given periods of kernel development, and that this past month has had somewhat of a memory management theme. This certainly seems to be the case. But even more specifically, this month’s theme would seem to actually be that of NUMA (NonUniform Memory Access) development. NUMA is all about dealing with the reality that as SMP (symmetric multiprocessing, the kind used in
your computer to provide many CPU cores for the operating system) scales, it does not necessarily do so in a linear fashion. The truth is that certain memory DIMM banks are ‘closer’ to certain CPU cores. In an SMP system, every CPU can access every memory location in the system, which is typically ‘coherent’, meaning that the underlying hardware takes care of ensuring that memory locations cached internally by other CPUs are updated when a different CPU updates that same memory. But accessing a given memory location can be more expensive in terms of latency if it is not ‘local’ to a given processor – ie if the underlying hardware must take more steps to reach a given location by routing access to that location through an inter-processor local bus. To an application (and a user) this manifests in terms of slower performance than could be achieved if the memory for an application were strictly local to a given processor. Linux handles NUMA in various ways. To a certain extent it can provide a level of transparent support, by ‘migrating’ memory for applications to be closer to the processors that are using it (copying the underlying memory locations to other locations more local to a given processor and updating the virtual memory translation tables accordingly), but special tools have been written to allow administrators to be more specific about how a given application should manage its memory. Mel Gorman, famous for writing the definitive book on Linux memory management, has been working on NUMA scalability problems recently, and both he and
Opinion
The kernel column – Jon Masters
open source
“This month’s theme would seem to actually be that of NUMA” Johannes Weiner have posted patches that aim to better handle NUMA hinting and automatic reconciliation of memory closer to where it is used. Beyond NUMA, other memory-scalabilityrelated topics were discussed at length over the past month. Dave Chinner posted patches that convert the kernel inode cache (an internal kernel data structure responsible for caching metadata about mounted file systems) to use Paul McKenney’s famous (and IBM patented) RCU (read-copy-update) memory access optimisation. This allows the inode cache to scale more easily when Linux is run on very large NUMA or non-NUMA systems without using a previously global lock. Also scale related, Andy Lutomirski (of a large banking corporation) began a discussion concerning a new MADV_WILLWRITE flag that can be used when setting up memory mappings to inform the kernel that a given range of virtual memory will definitely be written to later. This allows the kernel to forgo its usual ‘copy on write’ (deferred allocation of new virtual memory regions until the first use) and instead take the (slightly more expensive) up-front hit of directly allocating the memory in order to save a latency hit later when that memory is touched by a running application. Memory management contributions this month also included a virtualisation optimisation from Martin Schwidefsky (of the IBM s390 team), who provided patches that allow guest instances to inform a host hypervisor that they are not using pages (the fundamental unit of memory chunks that the system uses to manage memory used by applications, virtual machines, the kernel itself and so on), which are automatically discarded whenever they would be swapped out to disk, and reinitialised when required later. Still further in the realm of memory management, Michal Hocko provided a patch that prevents the OOM (out of
memory) killer from being triggered on kernel allocation faults – the kernel should itself be able to handle memory allocations failures, or use dedicated reserve memory pools.
power capping framework
Beyond memory management, Srinvas Pandruvada (Intel) posted an initial RFC (Request for Comments) patch – the first stage in developing a new idea into code – series that implemented an in-kernel power capping framework. The basic idea is to allow a given system (platform) to specify how much power is available and for a precise cap to be enforced across a variety of CPU and non-CPU devices installed within the system. Using this mechanism, it is hoped to provide certain limits (such as the amount of power that can be delivered by a battery, or power utility, or the amount of power available if a battery is intended to last for a certain time period, and so on) and for devices to provide various performance tradeoffs within those constraints. But by providing an overall framework, it is possible for the kernel to make whole-system decisions that are not isolated to a given subsystem or to a given device in isolation. Initial comments on the power capping framework were favourable, although Greg Kroah-Hartman (now of the Linux Foundation) suggested that he wanted to see actual users of the framework before it is merged – by which he meant that he really wanted to see Intel post code for its own CPUs and platforms to make use of this framework. This seems to be in progress, since comments on the mailing list implied that such patches exist and have been reviewed by at least a subset of developers to this point. It will be interesting to see where this development leads, especially in the mobile space, and also when applied to other architectures.
ongoing development
Beyond NUMA and virtual memory, development has included an initial implementation of UEFI boot stub support for 32-bit ARM systems (which previously used a less standard embedded bootloader called U-Boot), a patch to allow perprocess control of transparent huge pages, a fix for PCIe reinitialisation when performing a kernel crash dump (and kexec into a kernel that must not have outstanding DMA operations pending on boot), and optimisations from Andi Kleen for the kernel build system known as kbuild. Andi’s patches convert the ‘kallsyms’ stage of kernel compilation into a single-pass, rather than being the multi-stage process that has been the case. This is the point during kernel compilation at which all of the symbols (function names provided by the kernel for internal and modular use) are resolved during the final linking of the kernel image. Andi has a series of cute hacks proposed for dramatically speeding up the process. In this month’s kernel announcements, Ted Ts’o reminded everyone that the 2013 Linux Kernel Summit is coming up in Edinburgh next month, and that there will be places reserved for those who are strictly hobbyist developers. This seems to have been a late decision since the deadline for submissions came and went quickly, but it will be worth keeping an eye on this next year, in case such an invitation is repeated. Beyond the Kernel Summit, there will be a number of kernel and nonkernel developers present at the Linux Plumbers Conference in New Orleans, at which several other mini-conferences will run concurrently. These include an ACPI/Power Management gathering, a PCI mini-conference and this year’s Linux Security Summit, among many others.
23 www.linuxuser.co.uk
Feature
Android Studio
Get started with
Android Studio Get your head around the early access preview of the next stage in Android development
A
nnounced earlier this year at Google I/O, Android Studio will be the replacement for the current Android development solution in Eclipse. This Eclipse setup is part of the Android Developer Tools, and takes advantage of the IDE’s Java-based development environment and plug-in support to currently create the development portion of Android apps. With the rest of the SDK, virtual devices can be created and used to test code before deployment onto real devices. The Android Studio aims to be an all-in-one solution for development and testing. Coming with the entire SDK, and not even requiring traditional compiling or installation on Linux, Android Studio is an incredible easy and quick way to get straight into coding your apps – whether you’re an experienced Android/Java dev working directly in code, or a novice looking to use the graphical, drag-and-drop approach to app creation. Android Studio is still currently in beta as a free early preview – we’ll cover some of the current quirks with the software. However, it’s a definitely a great way to get used to using the software before the final switchover occurs. We’ll also cover how you can export from Eclipse if you’ve already been using that, and how to then distribute your app once it’s finally complete.
24 www.linuxuser.co.uk
Android Studio
Get to grips with the new IDE for Android development
Feature
25 www.linuxuser.co.uk
Feature
Android Studio
Introducing Android Studio The new Android Studio is based on IntelliJ IDEA, a much smaller and streamlined IDE than Eclipse. Android Studio utilises its features to create an all-in-one Android development environment with a smart visual view that is great for people just getting into app development, and the standard text editor for those who know their way around Java and the Android API.
Installation is very simple as well – the files contain both the IDE and the SDK you’ll need to create your apps, so you won’t need to install the SDK via ADT separately like you will have done with Eclipse if you’ve developed for Android in the past. The Studio allows you to edit how the SDKs are utilised, how imports and exports work, and even has a plug-in manager.
■ New Project Start a new project by going through the handy Android new project wizard, selecting API levels and setting the icon ■ Recent Projects The recent projects view allows you to go straight into any project you wish to work on without having to look in the Open Project prompt
■ Docs and How-Tos Some tips and how-tos for IntelliJ, the base of Android Studio, are available straight from the welcome screen
■ Check for updates
Android Studio has its own update manager, allowing you to keep it up to date. It will notify you when a newer version is available
Installation
01
Download
Head over to the Android Developers website and grab yourself a copy of the early access preview to Android Studio from here: bit.ly/1bWrFXz You’ll need to then extract the contents – put it in a folder you have easy access to, as you need to run the Studio from the folder.
26 www.linuxuser.co.uk
02
Runtime Use cd in the terminal to navigate to
03
the folder you just created, specifically the First run android-studio/bin/ directory. To launch If this is your first time using Android Android Studio, enter into the terminal: Studio, you can safely ignore the prompt to import any settings. Otherwise, find the $ ./studio.sh If it asks about your Java settings, for file you created from before, or your old now press Enter, as we can fix that later if it installation folder, to get previous settings. We can import from Eclipse later on. becomes a problem.
Android Studio
Get to grips with the new IDE for Android development
FEATURE
Helpful shortcuts
Key features
Android Studio has handy, configurable keyboard shortcuts. Here’s a list of the most common to get you started…
Development views As the Java for the Android apps is split into visual and function elements, you can use a visual WYSIWIG editor for the XML code, allowing you to place buttons and text and other UI elements wherever you want on the screen. The text part is a full IDE, with code navigation, debugging, syntax highlighting and a smart code analyser that warns you of any obvious issues with what you’ve written.
Command look-up (autocomplete command name)
+
Ctrl
+
Shift
A
Project quick fix
+
Alt
Enter
Reformat code
+
Ctrl
+
Alt
L
Show docs for selected API
+
Ctrl Virtual devices Test your apps on a virtual device that you define, either by using settings for the main Nexus devices, or by using generic settings for differentsized Android phones or tablets. You can edit the system settings to limit or increase RAM, and even allow it to use the system GPU if you need it to. It’s not perfect, but it’s a great way to quickly test functionality.
Q
Show parameters for selected method
+
Ctrl
P
Generate method
+
Alt
Insert
Navigate open tabs
+ +
Alt Alt
Jump to source
F4 Delete line
+
Ctrl
Y
Search by symbol name
+
Ctrl
+
Alt
+
Shift
N
Build
Eclipse migration
Debugging
For existing Android developers, it’s easy to migrate from Eclipse and the ADT to Android Studio. The tools are already in place to export the necessary files from Eclipse, and Android Studio includes a handy feature that allows you to then import them, and any other Studio file. This is helpful now and will be essential when it replaces Eclipse.
A specific debugger can analyse the code as it runs and give you a full rundown of what’s happening when, allowing you to go through the logs and figure out any possible issues, or work out where problems are already occurring. You can pause the program at any point to isolate issues, instead of having to search through code as it continues to run.
+
Ctrl
F9
Build and run
+
Shift
F10
Toggle project visibility
Alt
+
1
27 www.linuxuser.co.uk
Feature
Android Studio
FAQ
Since Android Studio is still under some development, there are some bugs that still need to be ironed out. Here are a couple of known issues with Android Studio that you can work around.
Create a new project Android Studio offers a quick way to start your app: name, package name, theme, advanced icon options and the ability to select between different Android APIs. The latter allows you
to put together an app with specific versions of Android in mind, making newer API objects available for some. We’ll look at a basic setup first so we can make our Hello World app later.
01
Create New Project
02
Set image
03
Activity selector
04
Activity name
05
Project structure
06
Views
1) Error: Gradle project refresh failed Android Studio has a newer version of Gradle that has some backward compatibility issues. These can be fixed by first clicking the link to search build.gradle from the error dialog. From there, doubleclick the line under the build.gradle usage, opening the project build.gradle file. Edit the classpath to change the gradle version to “0.5.+”. Finally, save the file and rebuild your project. 2) Error: Failed to import Gradle project This can be a problem after upgrading Android Studio, as the project files may point towards an SDK element that no longer exists. To fix this, you’ll need to quickly install the Android Support Repository. First, open Android SDK Manager, find and expand Extras, and install Android Support Repository.
From the Welcome screen, click on Create New Project to open up the dialog window. Name the project HelloWorld – we’ll be creating the code for that shortly. Leave all the other default settings – they relate to what versions of Android will be able to use the app.
For now, we’ll just use the blank activity. When making other apps, this activity selector can help you choose how the app with behave – such as being full-screen or having a multi-column flow to view several items.
Here you can set your image, giving you a nice preview of how it will look on different types of devices. You can set padding, background colour and more to your app icon. For now, click Next, as we don’t need an icon for the sample project.
Here you can name the activity and customise the way it works, with tabs, dropdown menus and other actions. For now, we’ll leave this as is. Hit Finish and the project will be built. It may take a little while to do this.
Migration
It’s quite easy to export existing projects from Eclipse and then import them into Android Studio. First of all, make sure your ADT plug-in is up to date, then select File and Export. Open Android from there and select Generate Gradle files, then your projects to export before clicking Finish. In the basic Android Studio window, click on ‘Import project’, find the build.gradle file and then press OK at the pop-up. You should then be able to start development on the project within Android Studio.
28 www.linuxuser.co.uk
Android Studio will create a lot of files for what will be a simple app – luckily we just need to pay attention to MainActivity.java and Main_Activity.xml for the moment. The interface is also initially set up to allow for WYSIWYG visual development.
The default view is the visual development interface – it includes elements down the side that allow you to drag and drop functionality. You can also click the Text tab at the bottom to get straight into the Java code, with a live preview updating as you type.
Android Studio
Get to grips with the new IDE for Android development
FEATURE
A view to develop The two views for Android Studio serve different purposes – and people. The default visual view allows you to drag and drop functions onto the interface, with guidelines to make sure you keep them aligned with other UI elements. This then adds the elements to the code, so you can edit them further in there.
■ Layouts & widgets Drag and drop different layouts and functions to create your app visually – a great way for novices to get started
■ Live preview See a live preview of the way your app will look, even while using the text mode
The text view works like any good IDE, with a hierarchical view of code, code navigation tools, autocomplete and, of course, proper syntax highlighting. Whichever view you use is up to you – however, you will ultimately have more control with the text view.
■ Toolbar Quick access to preview and debugging tools, along with the ability to change the theme, activity type and the device preview
■ Component tree Even in the visual view, you can break down the code in the individual components to see how they work
Useful resources There are plenty of online resources you can use to help you with your development – whether you need to figure out a more elegant method to create a specific function, or need to wrestle with some bugs. Here’s a list of some of the places you should visit. Android Developer Site – developer.android.com As well as the place to get Android Studio, the
Developer site has regular posts on interesting projects, updates to the SDK, and some troubleshooting tips.
Support for IntelliJ IDEA, the base for Android Studio. Some of the support questions may be more suited to the IntelliJ side of things.
Stack Overflow – stackoverflow.com
XDA Developers – www.xda-developers.com/
One of the best places on the internet to get advice on an issue – either by searching for a related question or submitting your own. IntelliJ IDEA Support – intellij-support.jetbrains.com/home
A community built around smartphone development, the XDA forum is a good way to find out about Android-specific tips and tools, and even to promote your application.
29 www.linuxuser.co.uk
Feature
Android Studio
Make your first application Creating your first app in Android Studio is easy, and we’ll start with the simplest one of all – the classic Hello World. All basic apps, when created, have Hello World already displayed
01
Visually
To do it visually, simply select Plain TextView, and place it on the display. Doubleclick it and type in ‘Hello World!’. All static text can be placed this way onto the interface, and it will create the necessary code in the text view as well.
02
Relative text
android:layout_height=”wrap_content” android:text=”Hello World!” android:id=”@+id/textView”/> The layout variables can also be set as fixed pixel width/height.
04
Quit button
05
Imported
The file we’re editing now is just the display. To do something, we need to start editing the .java file. While we’re still in the .xml, place a button and give it the caption ‘Quit’. Open up MainActivity.java from the Projects column before we continue.
The relative layout should be included in the code by default. Create a text parameter by opening up triangle brackets with:
Options>Advanced. In the Java Options section, select Class Path and then Add Archive. The file you need is located at: /usr/share/java/mysql-connector-java.jar. Select it and restart LibreOffice.
Connectivity between Base and MySQL makes use of a Java class. Type sudo apt-get install default-jdk to install the Java runtime. Type sudo apt-get install libmysql-java to install the needed additional Java classes.
35 www.linuxuser.co.uk
Tutorial
08
Make a small business database with LibreOffice
Connect the database
We now need to connect our frontend (LibreOffice) to the back-end (MySQL) of our database. Start LibreOffice and launch the Base module. In the dialog that pops up, select ‘Connect to an existing database’. From the dropdown menu below this, select MySQL as the database type.
09
Configure Base
On the next page, select ‘Connect using JDBC’. On the next page, click on ‘Test class’ to ensure that the Java RT is working. Now enter the name of the database that we created, customer, and enter localhost into the Server field. On the next page, add Root as the username and tick ‘Password required’. Now click on the ‘Test connection’ button and enter the root MySQL password, when asked for it, to test the connection between LibreOffice and the local MySQL server. Presuming that this completes without errors, click on Next.
10
Save the database
You can accept the defaults on the next page, so click on Finish. When prompted, give the database a name and save it. Remember that this file contains the connection information for access to our MySQL database – it doesn’t contain the actual records.
11
Create fields from a template
You may want to create a custom set of fields for your customer records, but to save time we’re going to use the one of the templates that is built into Base. Select Tables from the side menu and then ‘Use Wizard to Create Table’. Using the Sample tables pull-down menu, select Customers. Use the >> button to copy all of them across. On the next page, you can tweak the fields that you have included and add new ones. Select the defaults on the next two pages and then click on Finish.
12
Create form from template
13
Finalise form
Select Forms from the sidebar. Click on ‘Use Wizard to Create Form’ in the Tasks window. In the table wizard, click on the >> symbol to copy across all of the fields in the database.
Accept the defaults in sections 2, 3 and 4, but select the first arrangement icon in section 5, ‘Arrange controls’. You should now see a preview of our entry form in the main window. Select defaults on the other sections and then click on Finish.
Remember that this file contains 14 Test data entry the connection information for access to our MySQL database – it doesn’t contain the actual records
To enter data into the database, use the form that we created. Select Forms in the sidebar and then double-click on the name of the form in the main window. This brings up the GUI recordediting interface. The form can still be tweaked and edited by right-clicking on its name in the main window.
36 www.linuxuser.co.uk
Make a small business database with LibreOffice
Use LibreOffice and MySQL to create a customer database
TUTORIal
18
align the fields
19
Create a new database user
20
Redistribute the database
The field names from our imported data don’t quite match those of the database and so we need to use the second page of the wizard to line them up. To do so, click on a field name and use the up and down icons in the other list to create the correct attachments. Then click on Create.
15
Export your contacts from Gmail
16
Clean up the data and create a key
Switch from the Gmail contacts view using the pull-down menu in the top-left corner, underneath the Gmail logo. From here, click on the More icon pull-down menu and select Export… Click on Export.
Start a new spreadsheet and open the CSV file that you exported from Gmail. Use Ctrl-mousewheel zooming to get an overview. Typically, a lot of the fields will be completely blank, so select these columns (click on the column letter at the top of the window) and remove them (Edit>Delete cells). We have to create a key for each record. Label a column ID. Select the first cell in the column and then select the final cell by Shift-clicking on it. Use the fill feature (Edit>Fill>Series).
17
Import the data into Base
When you’ve cleaned up the spreadsheet, select the data (including the column headers) by clicking on the top-left cell and then Shiftclicking on the bottom-right cell. Right-click and select Copy. Select Tables from the side menu of the Base module. From here, right-click on the customers table and click on Paste. This should bring up the import wizard. Select ‘Append data’ and ‘Use first line as column names’ options, and click on Next.
To access the database from more than one machine, you must create additional users. Log back into phpMyAdmin, click on the Users tab and select ‘Add user’. From here, create a new user with the name and password of your choosing and make a note of it. Click on ‘Check all’ in the ‘Global privileges’ section.
In the Base module, re-save the database under a new name. In this new version of the file, we have to alter a few details. Select Edit>Database>Properties and enter the name of the new database user. Click on the Additional Settings tab and enter the IP address of the machine with the MySQL database.
37 www.linuxuser.co.uk
Tutorial
Write LaTeX documents with LyX
The toolbar gives you shortcuts for items like lists, images, tables and maths formulae
You can always see what the actual LaTeX source code looks like for your document. Nothing is hidden from you in LyX
The main window is where you type your text. This is also where your text gets rendered
You can view what your document will look like by rendering it to a PDF file. Don’t forget to refresh it after making edits
Write LaTeX documents with LyX
LyX lets you to get your document written without having to deal with LaTeX’s steep learning curve. Here’s how… Advisor
Joey Bernard As a true renaissance
man, he splits his time between building furniture, helping researchers with scientific computing problems and writing Android apps. When the kids let him have some time, that is
Resources
LyX: www..lyx.org LaTeX: www.latex-project.org
38 www.linuxuser.co.uk
LaTeX is a typesetting system that gives you full control over how everything in your document is rendered. The problem is its really steep learning curve. One option is to use a basic text editor and learn all the markup you need for your document. The other option is to use an application that wraps the markup to some degree. LyX does this very nicely. While a fully WYSIWYG editor for LaTeX doesn’t make sense (since your doc isn’t fully rendered until sent to an output device), LyX does provide a pseudo-WYSIWYG interface where you can see how different regions will be rendered.
In this tutorial, you’ll learn how to create a new document and create various content sections, like images, tables and lists. You will also learn how to use various document settings, like the document class, to control the overall options used during document rendering. Since LaTeX is a typesetting system, LyX will let you output to several formats, like PDF, Postscript, HTML, plain text and OpenDocument files. Regardless of whether you are writing a book, a journal article or a set of presentation slides, LyX will help you get your work done with minimal fuss.
Write LaTeX docs with LyX
Use LyX to lay out your LaTeX documents with ease
TUTORIal
01
Installation
The first step is to get LyX installed on your system. Most distributions should have a package available. For your non-Linux friends, there are binaries available for Windows, Mac OS X, and even OS/2 and Haiku. As always, you can download the source code and build from scratch in the worst-case scenario.
04 02
Opening a new document
03
Setting the document class
When you first start up LyX, the main window opens up with a splash screen image displayed. You actually have to tell LyX that you want to start a new document before you can start writing. You can start a new document by clicking the menu item File>New.
Text layout
While the document class sets the defaults for your document, you still have full control to change anything in the document. The Text Layout option lets you change the indentation size to a custom increment. The vertical spacing and line spacing can also be customised. You also have the option to make your document two-column here.
05
Page layout
LaTeX figures out the actual rendering of your document based on a page layout. You can select one of a number of standard page formats, or set a custom page size. You can set whether the page is oriented as either portrait or landscape mode.
06
Numbering and the TOC
07
View source
Documents are broken down into several nested subregions. How these regions are handled in the layout is decided by the document class. You can also set whether these regions are numbered or not, and whether they show up in the generated table of contents. Now that your document is set up, you can start typing. LyX gives you a pseudoWYSISYG display of the text. But LaTeX is a purely text-based markup language, so you can always see the actual source code to verify what LyX is putting into your document. To see the source, just click on the menu item View>View Source.
Many of the layout properties for your document are set to defaults based on the class of your document. You can set this by clicking the menu item Document>Settings. This will pop up a dialog window that we’ll use for the next few steps. The drop-down list will give you a very full list of possible document classes.
39 www.linuxuser.co.uk
Tutorial
Write LaTeX documents with LyX
Many of the layout properties for your document are set to defaults based on the class of your document
08
Title
The first item your document will need is a title. In LaTeX, you need to worry about what a particular piece of text is supposed to be, not what it will look like. So at this stage, you can type in your title text. To identify it as a title, you can click the drop-down at the top right of the toolbar and select Title.
10
Dates
11
Abstracts
You should have noticed that there is a date option in the drop-down. You don’t need to use this if you don’t wish to. LyX will automatically add the current date when you actually render the document into an output format.
If you are writing an article, or a report, you may need to include an abstract of the subject matter being covered. If you have already written your abstract, you can highlight the region with your mouse and select Abstract from the dropdown. This adds the title ‘Abstract’ and changes the format of the text.
09
Author
Hitting Enter will give you a new line, with the type back to Standard. You can here enter your name and set the type to Author, again with the drop-down at the top right of the toolbar. Where and how this will get rendered depends on the document class and the output format.
40 www.linuxuser.co.uk
12
Sections
13
Mathematical formulae
In most documents, you will probably want to break the text down into sections, and possibly subsections. Sections are defined by giving them a title. For instance, if your first section is going to be an introduction, then you would type the section title as ‘Introduction’ and set the type in the drop-down to Section. The actual text for this section would be set to the type Standard.
Mathematical formulae are always a problem area in document typesetting. Many people in the sciences first move to LaTeX because of the ability to fully control equations by explicitly laying out all of the elements. However, for more complex equations, this can still be confusing. Therefore LyX provides an equation writer tools that helps you create the LaTeX required to lay out your formula in your document. You can set the type of maths to be either inline with your text, or to be centred and displayed on its own.
Write LaTeX docs with LyX
Use LyX to lay out your LaTeX documents with ease
TUTORIal
14
lists
15
Tables
There are several different types of lists available for your use. Both numbered and unnumbered lists are available as buttons on the main toolbar. To start a list, click on one of the buttons and start typing the first item. Hitting the Enter key will give you a new item to enter. Hitting Enter on an empty item will drop you out of the list section.
You can add a table by clicking on the menu item Insert>Table. A dialog will appear where you can set the number of rows and columns. The first row is set aside as a header for the columns, but you can change this in the LaTeX source.
(pdflatex)]. There are also other viewing options, in case you want to render your document using other methods.
16
Images
Clicking on the menu item Insert>Graphics opens a dialog window where you can select an image file to insert into your document. You can either set a scaling factor, or an explicit width and height for its display. You can also rotate your image through X degrees. You also have the option to control if and how the image gets clipped to a bounding box.
17
provided by your LaTeX installation. This is defined in the Document Settings dialog, where you can set the family, encoding and fonts to use for Roman, Sans Serif and Typewriter.
20 18
Spellchecking
19
Fonts
Most people need help when it comes to making sure everything is spelled correctly. If you want to use the system default spellchecking engine, you can simply click on the menu item Tools>Spellchecker. You can change the engine being used by selecting it in the options window.
Previewing
Now that you have a bunch of content in your document, you probably want to get an idea of what it will look like once it is fully rendered. You can get LyX to generate a PDF for viewing by clicking the menu item View>View [PDF
Output formats
Now that you have your document finished, you will want to render it to some final output format. A common choice is either PostScript or PDF. This way, you know that it will look the same, regardless of who you give it to. But you have several other options available, too. You can output to HTML, rich text, plain text or even OpenDocument.
Linux users have had problems, traditionally, when dealing with fonts. This extends to applications like LaTeX. LyX includes options that help you correctly set up TrueType fonts, allowing you to use fonts other than those
41 www.linuxuser.co.uk
Tutorial
Unison synchronises files between computers – here we’re using the commandline version of the tool
Synchronise your files with Unison
This line indicates that Unison has nothing to sync at the moment, since no files have changed
Synchronise your files with Unison
This error message shows that there is a lock file that needs to be manually removed
The network connection is broken so Unison cannot synchronise files
Learn how to use the Unison command-line tool to synchronise files between computers quickly and reliably Advisor
Mihalis Tsoukalos has over 15 years of UNIX
system administration and programming experience and has been using Linux since 1993. He is also proficient in Oracle database administration, Cisco IOS, Cocoa and iOS programming. He always learns new things
Resources
Unison: www.cis.upenn.edu/~bcpierce/unison Two networked UNIX machines
42 www.linuxuser.co.uk
Unison is an open source file synchronisation tool for both text and binary files. It also has a GUI, but here we’re focusing on the only the command-line version because it’s quicker and gets the job done cleanly. Unison really shows its capabilities when you are working with more than one computer and you need synchronisation across all of them. Benjamin C Pierce led the creation of Unison at the University of Pennsylvania and it started life as a research project. It can be used through the
SSH service and works equally well on both UNIX (Linux, Mac OS X etc) and Windows machines. It should be apparent that Unison was inspired by the rsync utility. Unison differs from rsync in that the latter is a mirroring tool that needs to know in advance where the willing-to-keep versions of the files are, whereas Unison is a synchronisation tool that identifies the files that have been changed since the last sync process and decides the way that the changes are going to be propagated. In short, it’s smart.
Synchronise your files with Unison
Use the Unison command-line tool to sync files between computers
TUTORIal
01
Installing Unison
Most Linux distributions have Unison as a package ready for installation so that you do not have to compile it. At the time of writing, the current stable version of Unison is 2.40.102. Note that every machine that is part of the synchronisation process must have a copy of the command-line version of Unison installed. Additionally, this copy of Unison should be located inside one of the directories of the default PATH shell variable. Assuming you’re on a Debianbased system, simply type:
having a profile file does not prevent you from adding extra command-line options (although this article will not deal much with such options). In the rare event that you have troubles working with Unison, you may run it using the ‘-debug all’ command-line option so that you can better trace and resolve errors.
Synchronising files with another computer
The remote machine declaration starts with root = ssh://. There are rare occasions – usually when the user changed a file on both computers before synchronising – where Unison will not be able to determine whether a file or directory has changed on the local or the remote server. In such situations, Unison kindly asks for your help so that it will not mistakenly proceed using the wrong version.
unison -version
03
Unison can run from the command line without using any configuration files (profiles), but using a profile greatly simplifies its use –
04
$ unison articles
To find the version of Unison you are using, just type the following command:
Unison profile files
root = /Users/mtsouk root = /tmp
Given a profile name called articles.prf that is located inside the ~/.unison directory, you can tell Unison to use it by executing the following command:
sudo apt-get install unison
02
and is located inside the ~/.unison directory. The following two lines to work on the same machine, as they do not include a remote machine:
Synchronising files on the same computer
Although Unison was initially developed with synchronising files located on different machines in mind, you can synchronise files located on the same computer using one of the following two methods: • Using localhost as the remote machine name. • Using only local directory paths for both root = lines. Your advisor prefers the second way as it does not require the SSH server be up and running. The Unison profile file is called localFiles.prf
43 www.linuxuser.co.uk
Tutorial
Synchronise your files with Unison
07
The .unison directory
08
A more advanced Unison profile file
Most of the Unison housekeeping happens inside the .unison directory which resides in the user’s home directory. Unison keeps the following data there: • Its profile files including the default profile called default.prf. • Its backup files, if you decided to have central backups. • The status file for each used profile. • The Unison log file, should you tell it to keep one. You can put it anywhere you want but it is very convenient for it to reside there. • Unison creates lock files there, during synchronisations, which it deletes when finished. Note: It is not recommended to synchronise the whole .unison directory. Synchronising just the profiles is okay.
The contents of the advanced.prf file are as follows:
05
Running Unison using a simple profile file
Some basic things you need to know about Unison profiles are: • The default backup level is 2, which controls how many previous versions of each file are kept not including the current version of the file. So the default backup level keeps two backup copies of a file. The parameter for setting the backup level is called maxbackups. • You can tell Unison not to keep file backups by not adding any backup options inside a Unison profile file. • Unison profile files support Unicode characters. • The logfile option tells Unison to keep a log file called unison.log inside the .unison directory. • Sometimes, network problems prohibit file synchronisation, but Unison’s error messages are denoting the problem:
$ unison articles Contacting server... Connection closed by 109.xxx.yyy.zzz Fatal error: Lost connection with the server
06
Explaining the profile file
Lines starting with a # denote comments and are not processed any further. Exactly two lines must start with root =. They declare the machines (one is always the local
44 www.linuxuser.co.uk
computer) that are going to participate in the syncing process as well as the directories that are considered the root directories for each machine. After those important declarations, the actual directories that are going to be synced are listed. In this example we have just one directory (one path = line). Its full path is /Users/mtsouk/docs/ article/working for the local machine and /home/ mtsouk/docs/article/working for the machine called linode (an IP alias inside /etc/hosts). You can have as many path entries as you want. All files are synced except the ones that match the ignore option. You can also have as many ignore = lines as you want and can use regular expressions in them. The backup option tells Unison to keep backups of all files.
root = /Users/mtsouk root = ssh://linode//home/mtsouk times = true batch = true # Log file logfile = /Users/mtsouk/.unison/ unison.log # Paths to synchronize path = code/C # Backup files backup = Name * backuplocation = central maxbackups = 3 ignore = Name {Thumbs.db} ignore = Name {.DS_Store} ignore = Path {Samples.lnk}
Synchronise your files with Unison
Use the Unison command-line tool to sync files between computers
TUTORIal
3. Copy the contents of the ~/.ssh/id_rsa.pub file from your local server into the file ~/.ssh/authorized_keys found on the remote server. One way of doing it is by executing the following command:
$ cat ~/.ssh/id_rsa.pub | ssh linode 'cat >> .ssh/authorized_keys' The next time you try to log into the remote Linux server using SSH, you will be asked for the passphrase of step 1 for the last time. From now on, you can log into the remote Linux server by just typing ssh linode:
09
Explaining the advanced profile file • The times = true line tells Unison to
synchronise modification times. • The maxbackups = 3 line tells Unison to keep the current file version plus three backups of it. • The backup = Name * line tells Unison to back up every file. • The backuplocation = central, which is the default option, tells Unison to keep all backups in a central location. If neither the backupdir preference nor the environment variable UNISONBACKUPDIR are set, the .unison/backup directory is used as the backup location. If set to local, then all backups will be kept in the same directory as the original files. • The batch = true option is a little tricky and you should be careful with it as Unison will ask no questions at all and non-conflicting changes will be propagated whereas conflicts will be skipped. Nevertheless, it is an essential option if you want to use Unison as a cron job. • The ignore = Name {.DS_Store} line tells Unison to not synchronise files that end with .DS_Store.
10
Using SSH without giving a password
The single most time-saving thing you can do is to set up SSH so that you will not need to enter your password each time you want to synchronise your files and directories. The procedure is easy and involves the following three steps: 1. Run ssh-keygen -t rsa You will have to enter a passphrase twice, so please do remember the passphrase! Two new files are going to be created: ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub. 2. You may need to create a directory called .ssh on the remote server if it does not already exist.
$ ssh linode Linux (none) 3.9.3-x86_64-linode33 #1 SMP Mon May 20 10:22:57 EDT 2013 x86_64 . . . Last login: Wed Jul 31 18:46:23 2013 from ppp-94-64-21-97.home.otenet.gr mtsouk@li140-253:~$ The first time you log into the remote server without typing your password, the following informative message will be on the screen:
Identity added: /Users/mtsouk/.ssh/ id_rsa (/Users/mtsouk/.ssh/id_rsa)
11
12
Unison hints and tips
The first two or three times you use a new profile, double-check if everything works as expected. • You do not need to use every parameter that Unison supports, just the ones that will do your job! • You can troubleshoot Unison using the -debug all command-line option. It will generate lots of output useful for debugging. • The more you use Unison, the more you will understand its practicality. • You should be very careful with your backup options, especially maxbackups, as it can take up too much space on your computer. • You can use Unison to securely exchange files between computers. • If a Windows machine is involved in the synchronisation process, be careful with file and directory permissions. • For non-critical data files you may run Unison once a day, but for critical data you should run it more often. • Unison cannot replace regular backups! • When you are making a new profile, either start simple or use an existing one as a starting template. Add the extra functionality and features while making sure that you always have a working profile.
Two common Unison troubleshooting techniques
There are times when things do not work as expected. Unison offers you many options that can help you both find and solve problems. The first option to try is the -testserver option that just connects to the remote server and then exits without synchronising any files. The second thing to do is run the following command:
$ ssh $PATH'
remote.machine.domain
'echo
The aforementioned command let you see whether the PATH is the same as when you log in using ssh remote.machine.domain. If the problem is with the PATH, check if the option PermitUserEnvironment in /etc/ssh/ sshd_config is set to ‘no’ and change it to ‘yes’.
45 www.linuxuser.co.uk
Tutorial
Browse privately with Onion Pi
This isn’t a hardware hack – a spare SD card can be used for the Tor router, and other SD cards can be used for different functions without any problems
Using not much more than a Raspberry Pi, you can route one or more systems through a Tor-enabled access point, guaranteeing anonymity
Browse privately with Onion Pi
Hook into the internet just about anywhere there’s an internet connection – a relative’s house, hotel rooms and more
Turn your Raspberry Pi into a highly secure and very portable router to keep your system safe and your browsing anonymous, wherever you are Advisor
Rob Zwetsloot models
complex systems and is a web developer proficient in Python, Django and PHP. He loves to experiment with computing
Resources
A Raspberry Pi Raspbian: www.raspberrypi.org/downloads Compatible Wi-Fi adaptor: www.adafruit.com/products/814
46 www.linuxuser.co.uk
We showed you in issue 129 how to turn your Raspberry Pi into the ultimate portable wireless router, requiring very little power and giving you a wireless network wherever there’s the most basic of internet connections. What if it’s not enough to know you can search the web, though? What if you want to be wholly secure as you do it? Then it’s time to upgrade the router with Tor to protect your privacy on the internet. This ‘Onion Pi’, as dubbed by Adafruit, combines Raspbian and
Connect everything over a wireless network – no need to directly connect to the Pi with a cable
Tor to create and secure a wireless access point using just a Raspberry Pi. This project is fairly straightforward: after setting up the wireless access point, we install Tor and do some basic setup tasks so that it routes traffic properly, and securely. This will keep you anonymous online – a handy feature in a time of privacy concerns all around the web. When the Pi is not connected to the internet, it should still function as a wireless router, allowing at the very least a wireless LAN in your location.
Browse privately with Onion Pi
Turn your Raspberry Pi into a Tor-enabled wireless router
TUTORIAL
06
Set up DHCP
Now we need to configure the DHCP server. Edit the configuration file with:
$ sudo nano /etc/dhcp/dhcpd.conf And start by putting a # in front of the two option domain-name entries, then remove the # in front of authoritative, seven lines down.
07
Server address
At the end of the configuration file, add the following:
01
Install Raspbian
Raspbian is the Raspberry Pi distro we’ll be using for the Onion Pi. Download the zip file, extract the image and then apply it to an SD card using:
$ dd bs=4M if=[version number]wheezy-raspbian.img of=/dev/[SD card location]
Plug your USB wireless adapter into the Pi and turn it back on. On another computer connected to the same network, open a terminal or type into the command line:
subnet 192.168.42.0 netmask 255.255.255.0 { range 192.168.42.10 192.168.42.50; option broadcast-address 192.168.42.255; option routers 192.168.42.1; default-lease-time 600; max-lease-time 7200; option domain-name "local"; option domain-name-servers 8.8.8.8, 8.8.4.4; }
$ ssh [user]@[IP address]
Save and exit.
04
SSH connection
Then enter the password for your Raspbian if it asks for it.
You can also use NOOBS to install Raspbian if you wish.
02
Set up Raspbian
Go through the initial Raspbian setup and make sure to turn on the SSH server, and to disable autoboot to desktop – this is unnecessary and will only use extra power. You can also tell it to fill up the rest of the card if there’s room for it.
03
Pi IP
We’ll be accessing your Raspberry Pi via SSH to set it up. To do this we need to know its IP address – you can find it by typing ifconfig into the command line. Make a note of it and turn off your Pi.
05
Install DHCP
To make life easier for any system connecting to the Pi access point, we need to install a DHCP server to it. We do this with:
$ sudo apt-get install hostapd iscdhcp-server DHCP will automatically assign IP addresses to network-attached devices, meaning you won’t need static IPs.
08
DHCP server
Edit the server configuration files so that it’s set to work in conjunction with the wireless adaptor:
$ sudo nano /etc/default/isc-dhcpserver Scroll to INTERFACES and change it to:
INTERFACES="wlan0"
47 www.linuxuser.co.uk
Tutorial
09
Browse privately with Onion Pi
14
Incoming Wi-Fi
We need to set up the Wi-Fi adaptor to be both static and accept incoming signals. First:
Network addressing
Setting up a NAT will allow multiple clients to connect. To do this, run:
$ sudo nano /etc/network/interfaces
$ sudo nano /etc/sysctl.conf
Put a # in front of iface wlan0 and following lines with wpa roam, iface default and any other affecting wlan0.
And add to the bottom of the file:
net.ipv4.ip_forward=1 Save this, and then finish by running:
$ sudo sh -c "echo 1 > /proc/sys/ net/ipv4/ip_forward"
10
Static IP
Now give the wireless interface a static IP – after the line allow-hotplug wlan0, enter the following:
iface wlan0 inet static address 192.168.42.1 netmask 255.255.255.0 Save and exit, and then set wlan0’s address with:
$ sudo ifconfig wlan0 192.168.42.1
12
WLAN configuration interface=wlan0 driver=rtl871xdrv ssid=[access point name] hw_mode=g channel=1 macaddr_acl=0 auth_algs=1 ignore_broadcast_ssid=0 wpa=2 wpa_passphrase=[password] wpa_key_mgmt=WPA-PSK wpa_pairwise=TKIP rsn_pairwise=CCMP
13
Hostapd
After saving and exiting, we need to edit hostapd to point it to this new file. Open it with:
$ sudo nano /etc/default/hostapd
11
WLAN creation
We need to create a new file that holds all the information for our wireless network. We are going to make it password protected so that only the people we want to can access it. To create the file, start with:
$ sudo conf
nano
/etc/hostapd/hostapd.
And then enter the text from the next step.
48 www.linuxuser.co.uk
15
IP tables
Run the following three commands to make sure the internet connection is forwarded correctly:
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
16
Apply configuration So that this still works after a reboot, type:
And then find the line #DAEMON_CONF="". Remove the #, and change it to:
$ sudo sh -c "iptables-save > /etc/ iptables.ipv4.nat"
DAEMON_CONF="/etc/hostapd/hostapd. conf"
Then add to the end of /etc/network/interfaces:
up iptables-restore < /etc/iptables. ipv4.nat
Browse privately with Onion Pi
Turn your Raspberry Pi into a Tor-enabled wireless router
TUTORIal
17
Wi-Fi final
Finally, set it up as a daemon so it runs at boot with the following commands:
sudo service hostapd start sudo service isc-dhcp-server start sudo update-rc.d hostapd enable sudo update-rc.d isc-dhcp-server enable And the wireless access point part will be finished.
18
Install Tor
After a reboot, we now need to install Tor. Do this simply with:
$ sudo apt-get install tor Once it’s installed, you’ll need to edit the Tor config file with:
$ sudo nano /etc/tor/torrc Follow the next step to add all the necessary information to it.
19
Tor configure Put this below the FAQ comment:
Log notice file /var/log/tor/ notices.log VirtualAddrNetwork 10.192.0.0/10 AutomapHostsSuffixes .onion,.exit AutomapHostsOnResolve 1 TransPort 9040 TransListenAddress 192.168.42.1 DNSPort 53 DNSListenAddress 192.168.42.1
If you want to keep SSH open to connect remotely, you’ll need to make an exception for that with:
$ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --dport 22 -j REDIRECT --to-ports 22
21
$ sudo sh -c "iptables-save > /etc/ iptables.ipv4.nat"
23
logging
We should create a log file in case you need to debug later. To do this, use these three commands:
$ sudo iptables -t nat -A PREROUTING -i wlan0 -p udp --dport 53 -j REDIRECT --to-ports 53
$ sudo touch /var/log/tor/notices. log $ sudo chown debian-tor /var/log/ tor/notices.log $ sudo chmod 644 /var/log/tor/ notices.log
And then route any TCP traffic with:
You can also check it with:
$ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --syn -j REDIRECT --to-ports 9040
$ ls -l /var/log/tor
Reroute Route all DNS traffic first, using:
24
Secure the router
Finally, we can activate the Tor service so that we can start using the access point securely with:
$ sudo service tor start You can check this if you wish with:
$ sudo service tor status
20
Table flush
We now need to flush the current IP tables so that we can get the routing to go through Tor. First of all, do:
$ sudo iptables -F $ sudo iptables -t nat -F
22
Check and save You can check the table setup with:
$ sudo iptables -t nat -L
To make it turn on at boot, you simple add it to rc.d with:
$ sudo update-rc.d tor enable
If you’re happy with it, save it to the NAT file like before with:
49 www.linuxuser.co.uk
Have you got unwanted monitors?
4,000 monitors needed for schools in Africa and Latin America • WEEE compliance guaranteed • 0% to landfill policy • All legal liabilities covered • Data wiped to UK military standards on all PCs
www.computeraid.org/donate 020 8361 5540
[email protected] Or find us on...
Registered Charity No. 1069256 Registered Company No. 3442679
The only Linux magazine for open source developers 100% FOSS focused » Features » Tutorials » Sample code
If it wasn’t open source, if it wasn’t GPL, there wouldn’t be the 20,000 plug-ins that have been written for it
Mark Little talking about WordPress
51
■ Mike Little celebrates receiving his outstanding contribution award
The other half of WordPress
With ten years under its belt, WordPress has firmly cemented its place in the hearts of bloggers. But what next for its co-founder Mike Little? David Crookes finds out… We are on a tram to Manchester, tapping away on our phone as we update one of our blogs, engrossed in the words and thinking little about the underlying code which is making all of this possible. Just as the last few words are written, the tram stops and we make our way up the escalator to Piccadilly Station about to meet the man who created the behemoth that is WordPress, the very blogging platform we had been using throughout the journey. Our meeting is being held at a railway station because Mike Little, who is from Stockport, is a busy man. He has already had one meeting before we shake hands and he is due to have
52 www.linuxuser.co.uk
another. We decide to sit in a coffee shop and he grabs a sandwich before telling us about his latest venture, which is – unsurprisingly given the software is used by 18.9 per cent of the top 10 million websites – very much related to WordPress. Little is currently working on a series of screencasts which showcase how WordPress can be used, explaining the fundamentals of it to encourage more people to get involved with blogging using the platform. It’s a slow process – “it takes about an hour to edit a minute’s footage,” he says – but he’s getting there. The idea is that the screencasts will form a paid-for online course. “I want them to be better than anything else that is out there, so they have to be done right,” he says. “There is a lot of rubbish
being made where you get someone just filming what they are doing and there are lots of ums and ahs and backtracking. My screencasts won’t be like that and people will certainly get value for money.” Little, who also teaches WordPress to classes face-to-face, continues to work with WordPress for professional clients too. After all, if he cannot produce a killer website for companies and organisations using the software, then nobody can. It is fitting, then, that he has worked with Downing Street on government sites. He appears most proud of a science-engagement site for children called I’m A Scientist, Get Me Out Of Here, which has spun off into I’m An Engineer. “I really enjoy doing these,” he says, explaining that his entire freelance
The other half of WordPress
The story of WordPress co-founder, Mark Little
LITTLE AND LINUX
■ Matt Mullenweg’s WordPress.com website is run by his company Automattic
career has been based around WordPress since he left full-time employment in 2008. WordPress has, therefore, shaped his life. In between mouthfuls, he tells us all about the WordPress project, barely pausing for breath, and there is little doubt that he is proud of the part he played in the initial years of WordPress’s inception. If there was any doubt, then the snazzy WordPress T-shirt he’s wearing at our meeting certainly puts paid to that.
domain that was up for renewal. Another programmer, Matt Mullenweg, also used b2/ cafelog, or, as it was more commonly known, b2, and he was concerned not only about Valdrighi’s disappearance but also about the lack of development of b2. He posted an article on his blog called The Blogging Software Dilemma which discussed how he had come to be using b2, primarily because it was the best of the bunch and something he was able to develop. Mike Little spotted this blog post and responded, asking Mullenweg if he would be interested in forking b2. Mullenweg said he would. The pair then got together, albeit remotely over the internet. “I’d communicated with Matt a couple of times before because he had a website with a really good gallery on it and I’d emailed him to ask him about the software,” recalls Little. “I’d actually communicated with him on the b2 forums as well, but I didn’t know he was the same person as the one who ran the gallery site. So when I saw his blog post about b2, I was interested. As it turned out, I was the only person who responded to his posting at the time.”
66%
Proportion of all WordPress sites that are in English
Before WordPress
Little has been programming professionally since 1990. His first six years were spent at an industrial software house called Pantek in Stockport, where he became responsible for a team of six and developed Visual Basic apps, learned C++ and gained experience in DOS, Windows SDK, assembler, Novell, TCP/IP and Microsoft networks. After leaving Pantek, he went on to work at various other software companies in the Greater Manchester area, but he was also an early blogger. He used software called b2/cafelog which had been written by Michel Valdrighi, the first Corsican blogger. A small community had built up around the software with around 2,000 active users on the forum, but development of b2/ cafelog suddenly ground to a halt when Valdrighi disappeared, leaving behind an app that had a few bugs, a couple of security issues and a
Version one
Mike Little is a long-term user of Linux. “I’ve been into Linux since the early days,” he recalls. “I could see that the openness, the ability to look at code, to modify [it]… was making better software… So I’ve always been passionate about it and I just absolutely think it’s the right thing to do.” He has fond memories of the days when Linux needed to be downloaded from FTP sites and stuck on several 3.5-inch floppies. “I’m going back to 1.something series kernels, so right from when there probably were only three or four distributions. Slackware was one but I don’t even think the likes of Debian had started in the early days. I remember Red Hat starting so I was into it before that.” Little had been involved in a project called DJGPP, which was a port of the GNU Compiler Collection onto 16-bit DOS. It is, he concedes, a ridiculous undertaking in hindsight, but it meant developer tools could be obtained on DOS without having to shell out money. “I think that was before I even got the world wide web, so at that time I had a modem but I was dialling up to bulletin boards… and getting software that way,” he explains. “And then when I got onto the internet I just kind of continued in that vein. So I’d already understood that this DJGPP was a port of these GNU tools so… I started looking at what they were and how I could use those and eventually got brave enough to partition my Windows machine and get Linux booted on that. Before that I used to boot it from a floppy… but yes, eventually I went over to dual booting into Linux and in 1998 I went over at home full-time to Linux… By the time I had a decent laptop, I started with Debian but my last three laptops have run Ubuntu.”
“That first release of WordPress had a couple of extra features I’d already created for b2”
Mullenweg and Little took the b2 software and began to work on it, fixing bugs and adding extra features. Then Valdrighi reappeared. He
53 www.linuxuser.co.uk
declared Mullenweg and Little’s software as the official successor to b2. “It turns out he’d been made redundant or something like that, had to give up his flat, gone back to his parents who didn’t even have internet and so on,” smiles Little. “So it was just the real world was much more important than a bit of software.” WordPress was thus born. It was January 2003. Little had been learning PHP and he felt he would work his way around b2 and make alterations. “No software is perfect, so it was just a case of thinking of things that we wanted to add to it,” says Little. “That first release of WordPress had a couple of extra features that I’d already created for b2 and the same for Matt: he’d had a couple of extra features that he’d added to b2. We’d shared these things on the forums and we kind of incorporated those into the core project. And yeah, we just took it from there.” One of the biggest changes was the introduction of Pages, which made a massive difference to WordPress and set it apart from its rivals at the time. It enabled WordPress to become more than blogging software. But the
TAKING INSPIRATION WordPress was not created in isolation. Although Little and Mullenweg worked on their own, with the expanse of the Atlantic ocean separating them, they did look at other blogging software being created at the time and they took inspiration from it. “There was definitely a lot of studying of rivals and playing catch-up, seeing what other people were doing,” Little says. “For a very long time there was a lot of blogging software and certainly you would see Blogger or Moveable Type with features that were cool and that we thought we might add to WordPress. Eventually it started going the other way round as well and these other apps were copying features that had been added to WordPress.”
54 www.linuxuser.co.uk
‘killer’ feature, Little says, came in the guise of hooks. “It meant you could write additions to the product without having to modify the code, and that was something that only really big grown-up software did, like some of the Java Stacks,” says Little. “This ability to actually change the way it behaves and add things to it without having to touch any of the core files was crucial. It’s still the hardest thing, I think, for new developers to get their heads round how it works, but I think that’s what made it so much easier.” He said it made the process of updating the software so much easier. “Prior to that, with anything like Drupal or any of those tools, whenever there was an update you were in danger of losing your changes because you’d modified original files to call your stuff or to add somebody else’s stuff in. You’d stick these function calls in there. It meant that as soon as you got the new version, if you just literally FTPed the new one over the top of the old one, you were going to lose your stuff and lose your modifications. So I think that was probably one of the key features and I don’t think it’s necessarily something that people appreciate because these days they just never see the problem. They update WordPress or they update the plug-ins and oh, it’s the new version, and nothing’s gone wrong. And yet before then you had to save those files you modified before you updated to the new software.” Mullenweg and Little continued to work on WordPress and the first release came in May of that year.
User-friendliness
Little and Mullenweg saw WordPress as an evolving product. Little worked on doing what he could to eliminate the ability to make mistakes, which he believes was a crucial step forward. “With the original b2, you could lose your settings,” he says. “But with WordPress, I added the config sample PHP and it just took that ability to make a mistake away. I took what was the original b2 settings file and created an Options interface for it. It was another file, you didn’t have to fiddle with it - you didn’t have to know how to edit these files and FTP them back up to your server and stuff like that. I think all those bits helped make it easier, made it harder to cock up and I think that all helped.” The need to be userfriendly was at the forefront of the minds of both developers. It was also important for them that WordPress, like b2, was an open source project. They loved that b2 operated under a General Public License, which meant they had the freedom to take the code, change it and distribute it to other people. They loved that it was free too. “I’ve always been a huge open source advocate,” says Little. “Matt was kind of new to the open source idea, but that was one of the reasons behind him choosing to use b2.” Little attributes the popularity of WordPress to its open source nature (“if it wasn’t open source, if it wasn’t GPL, there wouldn’t be the 20,000 plug-ins that have been written for it or the 10,000 themes,” he says) and it matters not to him that he has not made any direct money from WordPress. “Nobody does,” he says. “It’s
18.9% The percentage of the 10 million websites in the world that run WordPress
“If it wasn’t open source, if it wasn’t GPL, there wouldn’t be the 20,000 plug-ins that have been written for it”
The other half of WordPress
The story of WordPress co-founder, Mark Little
OUTSTAnDinG COnTRiBUTiOn TO DiGiTAL
n The original posting, The Blogging Software Dilemma, on Mat Mullenweg’s website – and Mike Little’s response
not sold and yes, I continued in my day job. Matt went [to work] for CNET, which had just started using WordPress for just a couple of things, and they actually had him doing a standard software development job. But they paid for him to work one day a week on WordPress itself and I think it was through working there that he met a guy called Tony Schneider, who had just sold some software to Yahoo! for £25m or something like. Matt had started Automattic at this point. Schneider later managed to help raise funding for him and joined the service that eventually became WordPress.com. But me, I continued in my day job up until 2008.”
Up until the first release, the development team consisted of just Mullenweg and Little, but five people were working on the project by the end of 2003 and it continued to grow after that. By the end of 2005, Little wasn’t able to stay involved. “I had issues at home, real-life issues which meant that I couldn’t spend the time working on this thing voluntarily,” he says. “It was still a very techy thing, a very geek thing, it wasn’t as good and as user-friendly as it has become today and it certainly wasn’t as prominent.” Cash, he insists, is something that does not motivate him, however. “I need to pay the bills and we need to eat,” he says. “I am the breadwinner of the house and I always have been, but I just wish I could put that side of things in place and do what interests me from that point on. There’s no doubting my passion for open source computing.” Indeed, he has read up on the philosophy behind the General Public License and he has devoured Richard Stallman’s story of how he created the free software movement. “I not only found it very inspiring,” he says, “it just seemed
4 billion
Carrying on
The number of page views of WordPress websites from April 2012 to April 2013
Little considered becoming part of WordPress.com, which is a blog web hosting service provider owned by Automattic, but this was prior to the funding and the company couldn’t afford to pay him a wage. Today it is financially supported via paid upgrades, VIP services and advertising and can pay wages, but in the early days there wasn’t much cash for this and the bulk of the fundraising efforts went on infrastructure.
At SASCon 2013, Mike Little was presented with the inaugural ‘Outstanding Contribution to Digital’ award for his part in co-founding and developing WordPress. The award, which was handed out at a conference at the new Manchester Metropolitan University Business School, acknowledged his instrumental role in setting up and developing the first version of WordPress, which has become the most popular content management system on the internet. Richard Gregory, one of SAScon’s founders, said: “There isn’t a single digital marketing agency that hasn’t made generated income by building WordPress websites, and we are honoured to be able to recognise him with this award.”
the morally right thing to do, for me. The fact is that, at the time, in the 1970s and 1980s, software companies were effectively creating artificial scarcity, based on intellectual ideas that were almost free to reproduce. Back then firms were making floppy disks and putting them in boxes and getting manuals printed and they were charging £500 or £600 for a product that had just £30 or £40 worth of materials. As things moved on, it got easier and easier to distribute software online where the distribution costs and the reproduction costs approach zero, so to then charge artificial amounts on top of that just seemed like the wrong thing to do.” He hopes that future generations go down the same path and release open source software. “But first, he says, “I think it is crucial that we just get kids learning how to code. There are some good moves in that direction and I love what is being done with the Raspberry Pi. Suddenly, a school can’t say it cannot afford to buy lots of computers to experiment with. These are costing £25 each. It’s a great time to be in computing and I feel it’s good to be putting back in and helping people learn.” Mike Little’s new website is at mikelittle.org but he also runs his company site, zed1.com.
55 www.linuxuser.co.uk
Tutorial
MakeextensionsforXBMCwithPython
Rating (only available for hosted plug-ins)
Current media selection
List of installed plug-ins
Configure launcher
Localised description string
Makeextensionsfor XBMCwithPython
Opens changelog for the plug-in
Python is the world’s most popular easy-to-use open source language. Learn how to use it to build your own features for XBMC, the world’s favourite FOSS media centre Advisor
KunalDeois a veteran open source developer. Currently he is leading two open source projects: WinOpen64 and KUN Wiki. He is also a KDE developer. He has contributed to many open source projects, including KDESolaris, Belenix and Openmoko
Resources
XBMC: www.xbmc.org/download Python2.7x PythonIDE(optional) XBMCisperhapsthemostimportantthingthat has ever happened in the open source media centre space. It started its life on the original Xbox videogames console and since then it has become the de facto software for multimedia aficionados. It also has been forked into many other successful media centre applications such as Boxee and Plex. XBMC has ultimately grown into a very powerful open source application with
56 www.linuxuser.co.uk
a solid community behind it. It supports almost all major platforms, including different hardware architectures. It is available for Linux, Windows, Mac OS X, Android, iOS and Raspberry Pi. In these pages we will learn to build extensions for XBMC. Extensions are a way of adding features to XBMC without having to learn the core of XBMC or alter that core in any way. One additional advantage is that XBMC uses Python as its scripting language, and this can be also used to build the extensions. This really helps new developers get involved in the project since Python is easy to learn compared to languages like C/C++ (from which the core of XBMC is made). XBMC supports various types of extensions (or Add-ons): Plugins, Programs and Skins. Plugins add features to XBMC. Depending on the type of feature, a plug-in will appear in the relevant media section of XBMC. For example, a YouTube plug-in would appear in the Videos section. Scripts/Programs are like mini-applications for XBMC. They appear in the Programs section. Skins are important since XBMC is a completely customisable application – you can change
the look and feel of just about every facet of the package. Depending upon which category your extension fits, you will have to create the extension directory accordingly. For example… Plug-ins: plugin.audio.ludaudi: An audio plug-in plugin.video.ludvidi: A video plug-in script.xxx.xxx: A program In this tutorial we will build an XBMC plug-in called LUD Entertainer. This plug-in will provide a nice way to watch videos from Reddit from within XBMC. Our plug-in will show various content such as trailers and documentaries from Reddit. We’ll also allow our users to add their own Subreddit. Each video can then be categorised as Hot, New, Top, Controversial etc. With this plug-in we will demonstrate how easy it is hook into XBMC’s built-in method to achieve a very high-quality user experience. Due to space limitations, we aren’t able to print the full code here. You are highly recommended to explore the cover disc to access the complete code, which has many other amazing functions.
Make extensions for XBMC with Python
Use Python to create a new plug-in for XBMC
TUTORIal
01
Preparingthedirectorystructure
As we have mentioned previously, each XBMC extension type follows a certain directory naming convention. In this case we are building a video plug-in, so the plug-in directory name would be plugin.video.ludlent. But that’s just the root directory name – we will need several other folders and files as well. The following describes the directory structure of LUD Linux Entertainer: plugin.video.ludent – Root Plugin directory |-- addon.xml |-- changelog.txt |-- default.py |-- icon.png |-- LICENSE.txt |-- README `-- resources |-- lib `-- settings.xml
02
Creatingaddon.xml
An addon.xml file needs to be created in the root of the extension directory. The addon.xml file contains the primary metadata from a XBMC extension. It contains overview, credits, version information and dependencies information about the extension. The root element of addon.xml is the element. It is defined as:
rest of the content is placed here Here, id is the identifier for the plug-in, so it should be unique among all the XBMC extensions, and id is also used for the directory name; version tells XBMC the extension version number, which helps in its ability to deliver automatic updates – XBMC follows the Major.Minor.Patch versioning convention; name is the English title of the plug-in. Note: Steps 3 to 5 cover entries that need to be added within the addon.xml file.
03
addingdependencyinformation
Dependency inside an extension is managed using the element.
In the above code we have added a dependency to a library called xbmc.python version 2.1. Currently it is added as a mandatory dependency. To make the dependency optional you will need to add optional="true"; eg In the above example we have added core dependency xbmc.python to 2.1.0 because it’s the version shipped with XBMC version Frodo 12.0 and 12.1 . If you were to add xbmc.python to 2.0 then it would only work in XBMC Eden 11.0 and not in the latest version. For the current version of XBMC 12.1, the following versions of core XBMC components are shipped: xbmc.python 2.1.0 xbmc.gui 4.0.0 xbmc.json 6.0.0 xbmc.metadata 2.1.0 xbmc.addon 12.0.0 In addition to xbmc.python we are also adding some third-party plug-ins as dependencies, such as plugin.video.youtube. These plug-ins will be installed automatically when we install plugin.video.ludent.
04
Settinguptheproviderand entrypoint
Our extension is supposed to provide the video content for XBMC. In order to convey that, we have to set up the following element:
video Here, the library attribute sets up the plug-in entry point. In this example default.py will be executed when the user activates the plug-in. The elements sets up the media type it provides. This also gets reflected in the placement of the plug-in. Since ours is a video plug-in, it will show up in the Videos section of XBMC.
05
Settingupplug-inmetadata
Metadata about the plug-in is provided in . The following are the important elements… : Most of the time, XBMC extensions are cross-platform compatible. However, if you depend on the native platform library that is only available on certain platforms then you will need to set the supported platforms here. Accepted values for the platform are: all, linux, osx, osx32, osx64, ios (Apple iOS) , windx (Windows DirectX), wingl (Windows OpenGL) and android. : This gives a brief description of the plug-in. Our example sets the language attribute as English, but you can use other languages too. : A detailed description of the plug-in. : Webpage where the plug-in is hosted. : Source code repository URL. If you are hosting your plug-in on GitHub, you can mention the repository URL here. : Discussion forum URL for your plug-in. : Author email. You can directly type email or use a bot-friendly email address like max at domain dot com.
06
Settingchangelog,icon,fanart andlicence
We need a few additional files in the plug-in directory… changelog.txt: You should list the changes made to your plug-in between releases. The changelog is visible from the XBMC UI. An example changelog: 0.0.1 - Initial Release 0.0.2 - Fixed Video Buffering Issue icon.png: This will represent the plug-in in the XBMC UI. It needs to be a non-transparent PNG file of size 256x256. fanart.jpg (optional): The fanart.jpg is rendered in the background if a user selects the plug-in in XBMC. The art needs to be rendered in HDTV formats, so its size can range from 1280x720 (720p) up to the maximum 1920x1080 (1080p).
57 www.linuxuser.co.uk
Tutorial
MakeextensionsforXBMCwithPython
License.txt: This file contains the licence of the distributed plug-in. The XBMC project recommends the use of the Creative Commons Attribution-ShareAlike 3.0 licence for skins, and GPL 2.0 for add-ons. However, most of the copyleft licences can be used. Note: For the purpose of packaging, extensions/ add-ons/themes/plug-ins are the same.
07
Providingsettingsfortheplug-in
Settings can be provided by the file resources/settings.xml. These are great for userconfigurable options. Partial:resources/settings.xml
Here, label defines the language id string which will then be used to display the label. id defines the name which will be used for programmatic access. type defines the data type you want to collect; it also affects the UI which will be displayed for the element. default defines the default value for the setting. You should always use a default value wherever possible to provide a better user experience.
58 www.linuxuser.co.uk
The following are a few important settings types that you can use… text: Used for basic string inputs. ipaddress: Used to collect internet addresses. number: Allows you enter a number. XBMC will also provide an on-screen numeric keyboard for the input. slider: This provides an elegant way to collect integer, float and percentage values. You can get the slider setting in the following format:
In the above example we are creating a slider with min range 1, max range 10 and step as 1. In the option field we are stating the data type we are interested in – we can also set option to "float" or "percent". bool: Provides bool selection in the form of on or off. file: Provides a way to input file paths. XBMC will provide a file browser to make the selection of file. If you are looking to make selection for a specific type of file you can use audio, video, image or executable instead of file. folder: Provides a way to browse for a folder… Example:
Here, source sets the start location for the folder, while option sets the write parameter for the application. sep & lsep: sep is used to draw a horizontal line in the setting dialog; lsep is used for drawing a horizontal line with text. They do not collect any input but are there for building better user interface elements…
08
Languagesupport
Language support is provided in the form of the strings.xml file located in resources/languages/[language name]. This approach is very similar to many large software projects, including Android, where static strings are never used.
resource/language/english/string.xml example:
Add subreddit Hot New Top Controversial Hour Day Week Month Year As you may have seen in the settings.xml example, all the labels are referring to string ids. You can have many other languages as well. Depending upon the language XBMC is running in, the correct language file will be loaded automatically. Post XBMC Frodo (12.1), strings.xml will be deprecated. Post Frodo, XBMC will be moved to a GNU gettext-based translation system; gettext uses PO files. You can use a tool called xbmc-xml2po to convert strings.xml into equivalent PO files.
09
Buildingdefault.py
Since our plug-in is small, it will all be contained inside default.py. If you are developing a more complex add-on then you can create supporting files in the same directory. If your library depends upon third-party libraries, you have two ways to go about it. You can either place the third-party libraries into the resources/lib folder; or bundle the library itself into a plug-in, then add that plug-in as the dependency in the addon.xml file. Our plug-in works with reddit.tv. This is the website from Reddit which contains trending videos shared by its readers. Videos posted on Reddit are actually sourced from YouTube, Vimeo and Dailymotion. We will be starting default.py with the following imports:
import urllib import urllib2 … import xbmcplugin
Make extensions for XBMC with Python
Use Python to create a new plug-in for XBMC
TUTORIal
import xbmcgui import xbmcaddon Apart from xbmcplugin, xbmcgui and xbmcaddon, the rest are all standard Python libraries which are available on PyPI (Python Package Index) via pip. You will not need to install any library yourself since the Python runtime for XBMC has all the components built in. urllib and urllib2 help in HTTP communication. socket is used for network I/O; re is used for regular expression matching; sqlite3 is the Python module for accessing an SQLite embedded database; xbmcplugin, xbmcgui and xbmcaddon contain the XBMC-specific routine.
10
Initialising
During the initialisation process, we will be reading various settings from settings.xml. Settings can be read in the following way:
addon = xbmcaddon.Addon() filterRating = int(addon. getSetting("filterRating")) filterVoteThreshold = int(addon.getS etting("filterVoteThreshold")) In order to read settings of type bool you will need to do something like:
filter = addon.getSetting("filter") == "true" We are also setting the main URL, plug-in handle and the user agent for it:
pluginhandle = int(sys.argv[1]) urlMain = "http://www.reddit.com" userAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0" opener = urllib2.build_opener() opener.addheaders = [(‘User-Agent’, userAgent)]
11
Readinglocalisedstrings
As mentioned, XBMC uses strings.xml to serve up the text. In order to read those strings, you will need to use getLocalizedString.
translation = addon. getLocalizedString translation(30002) In this example, translation(30002) will return the string "Hot" when running in an English environment.
idFile
idPath
strFilename
playCount
lastPlayed
1
1
plugin://plugin.
2
2
plugin://plugin.
1
2013-08-07 22:42
3
2
plugin://plugin.
1
2013-08-08 00:09
4
2
plugin://plugin.
1
2013-08-08 00:55
5
2
plugin://plugin.
1
2013-08-08 00:58
dateadded
2013-08-06 23:47
12
Buildinghelperfunctions
In this step we will look at some of the important helper functions. getDbPath(): This returns the location of the SQLite database file for videos. XBMC stores library and playback information in SQLite DB files. There are separate databases for videos and music, located inside the .xbmc/userdata/ Database folder. We are concerned with the videos DB. It is prefixed with ‘MyVideos’…
def getDbPath(): path = xbmc. translatePath("special://userdata/ Database") files = os.listdir(path) latest = "" for file in files: if file[:8] == ‘MyVideos’ and file[-3:] == ‘.db’: if file > latest: latest = file return os.path.join(path, latest) getPlayCount(url): Once we have the database location, we can get the play count using a simple SQL query. The MyVideo database contains a table called files, which keeps a record of all the video files played in XBMC by filename. In this case it will be URL.
dbPath = getDbPath() conn = sqlite3.connect(dbPath) c = conn.cursor() def getPlayCount(url): c.execute(‘SELECT playCount FROM files WHERE strFilename=?’, [url]) result = c.fetchone() if result: result = result[0] if result: return int(result) return 0 return -1
The above table is an example of a files table. addSubreddit(): Our plug-in allows users to add their own Subreddit. This function takes the Subreddit input from the user, then saves it in the subreddits file inside the addon data folder. The following sets the subreddits file location:
subredditsFile = xbmc. translatePath("special://profile/ addon_data/"+addonID+"/subreddits") this translates into .xbmc/userdata/ addon_data/plugin.video.ludent/ subreddits def addSubreddit(): keyboard = xbmc.Keyboard(‘’, translation(30001)) keyboard.doModal() if keyboard.isConfirmed() and keyboard.getText(): subreddit = keyboard. getText() fh = open(subredditsFile, ‘a’) fh.write(subreddit+’\n’) fh.close() This function also demonstrates how to take a text input from the user. Here we are calling the Keyboard function with a text title. Once it detects the keyboard, it writes the input in the subreddits file with a newline character. getYoutubeUrl(id): When we locate a YouTube URL to play, we pass it on to the YouTube plug-in (plugin.video.youtube) to handle the playback. To do so, we need to call it in a certain format…
def getYoutubeUrl(id): url = "plugin://plugin. video.youtube/?path=/root/ video&action=play_video&videoid=" + id return url
59 www.linuxuser.co.uk
Tutorial
MakeextensionsforXBMCwithPython
On the same lines, we can build a function to place links as well… Similarly for Vimeo:
def getVimeoUrl(id): url = "plugin://plugin.video. vimeo/?path=/root/video&action=play_ video&videoid=" + id return url And for Dailymotion:
def getDailyMotionUrl(id): url = "plugin://plugin.video. dailymotion_com/?url=" + id + "&mode=playVideo" return url Once we have the video URL resolved into the respective plug-in, playing it is very simple:
def playVideo(url): listitem = xbmcgui. ListItem(path=url) xbmcplugin. setResolvedUrl(pluginhandle, True, listitem)
13
Populatingplug-incontentlisting
xbmcplugin contains various routines for handling the content listing inside the plug-ins UI. The first step is to create directory entries which can be selected from the XBMC UI. For this we will use a function called xbmcplugin.addDirectoryItem. For our convenience we will be abstracting addDirectoryItem to suit it to our purpose, so that we can set name, URL, mode, icon image and type easily.
def addDir(name, url, mode, iconimage, type=""): u = sys.argv[0]+"?url="+urllib. quote_plus(url)+"&mode="+str(mode)+" &type="+str(type) ok = True liz = xbmcgui.ListItem(name, iconImage="DefaultFolder.png", thumbnailImage=iconimage) liz.setInfo(type="Video", infoLabels={"Title": name}) ok = xbmcplugin. addDirectoryItem(handle=int(sys. argv[1]), url=u, listitem=liz, isFolder=True) return ok
60 www.linuxuser.co.uk
def addLink(name, url, mode, iconimage, description, date): u = sys.argv[0]+"?url="+urllib. quote_plus(url)+"&mode="+str(mode) ok = True liz = xbmcgui.ListItem(name, iconImage="DefaultVideo.png", thumbnailImage=iconimage) liz.setInfo(type="Video", infoLabels={"Title": name, "Plot": description, "Aired": date}) liz.setProperty(‘IsPlayable’, ‘true’) ok = xbmcplugin. addDirectoryItem(handle=int(sys. argv[1]), url=u, listitem=liz) return ok Based on the abstractions we have just created, we can create the base functions which will populate the content. But before we do that, let’s first understand how Reddit works. Most of the Reddit content filters are provided through something called Subreddits. This allows you to view discussions related to a particular topic. In our plug-in we are interested in showing videos; we also want to show trailers, documentaries etc. We access these using Subreddits. For example, for trailers it would be reddit.com/r/ trailers. For domains we can use /domain; for example, to get all the YouTube videos posted on Reddit, we will call reddit.com/domain/ youtube.com. Now you may ask what is the guarantee that this Subreddit will only list videos? The answer is that it may not. For that reason we scrape the site ourselves to find videos. More on this in the next step. The first base function we’ll define is index(). This is called when the user starts the plug-in.
def index(): defaultEntries = ["videos", "trailers", "documentaries", "music"] entries = defaultEntries[:] if os.path. exists(subredditsFile): fh = open(subredditsFile, ‘r’) content = fh.read() fh.close() spl = content.split(‘\n’) for i in range(0, len(spl), 1):
if spl[i]: subreddit = spl[i] entries. append(subreddit) entries.sort() for entry in entries: if entry in defaultEntries: addDir(entry.title(), "r/"+entry, ‘listSorting’, "") else: addDirR(entry.title(), "r/"+entry, ‘listSorting’, "") addDir("[ Vimeo.com ]", "domain/vimeo.com", ‘listSorting’, "") addDir("[ Youtu.be ]", "domain/ youtu.be", ‘listSorting’, "") addDir("[ Youtube.com ]", "domain/youtube.com", ‘listSorting’, "") addDir("[ Dailymotion.com ]", "domain/dailymotion.com", ‘listSorting’, "") addDir("[B]"+translation(30001)+" -[/B]", "", ‘addSubreddit’, "") xbmcplugin. endOfDirectory(pluginhandle) Here, the penultimate entry makes a call to addSubreddit. listSorting takes care of sorting out the data based on criteria such as Hot, New etc. It also calls in Reddit’s JSON function, which returns nice easy-to-parse JSON data. We have created a settings entry for all the sorting criteria. Based on what is set, we go ahead and build out the sorted list.
def listSorting(subreddit): if cat_hot: addDir(translation(30002), urlMain+"/"+subreddit+"/hot/. json?limit=100", ‘listVideos’, "") if cat_new: addDir(translation(30003), urlMain+"/"+subreddit+"/new/. json?limit=100", ‘listVideos’, "") if cat_top_d: addDir(translation(30004)+": "+translation(30007), urlMain+"/"+subreddit+"/ top/.json?limit=100&t=day", ‘listVideos’, "") xbmcplugin. endOfDirectory(pluginhandle)
Make extensions for XBMC with Python
Use Python to create a new plug-in for XBMC
TUTORIal
def listVideos(url): currentUrl = url xbmcplugin.setContent(pluginhandle, "episodes") content = opener.open(url).read() spl = content.split(‘"content"’) for i in range(1, len(spl), 1): entry = spl[i] try: match = re.compile(‘"title": "(.+?)"’, re.DOTALL).findall(entry) title = match[0].replace("&", "&") match = re.compile(‘"description": "(.+?)"’, re.DOTALL). findall(entry) description = match[0] match = re.compile(‘"created_utc": (.+?),’, re.DOTALL).findall(entry) downs = int(match[0].replace("}", "")) rating = int(ups*100/(ups+downs)) if filter and (ups+downs) > filterVoteThreshold and rating < filterRating: continue title = title+" ("+str(rating)+"%)" match = re.compile(‘"num_comments": (.+?),’, re.DOTALL). findall(entry) comments = match[0] description = dateTime+" | "+str(ups+downs)+" votes: "+str(rating)+"% Up | "+comments+" comments\n"+description match = re.compile(‘"thumbnail_url": "(.+?)"’, re.DOTALL). findall(entry) thumb = match[0] matchYoutube = re.compile(‘"url": "http://www.youtube.com/ watch\\?v=(.+?)"’, re.DOTALL).findall(entry) matchVimeo = re.compile(‘"url": "http://vimeo.com/(.+?)"’, re.DOTALL).findall(entry) url = "" if matchYoutube: url = getYoutubeUrl(matchYoutube[0]) elif matchVimeo: url = getVimeoUrl(matchVimeo[0].replace("#", "")) if url: addLink(title, url, ‘playVideo’, thumb, description, date) except: pass match = re.compile(‘"after": "(.+?)"’, re.DOTALL).findall(entry) xbmcplugin.endOfDirectory(pluginhandle) if forceViewMode: xbmc.executebuiltin(‘Container.SetViewMode(‘+viewMode+’)’)
14
Populatingtheepisodeview(listingvideos)
At this point we have the URL in hand, which returns JSON data; now we need to extract the data out of it which will make sense to us. By looking at the JSON data, you can see there’s a lot of interesting information present here. For example, url is set to youtube.com/watch?v=n4rTztvVx8E; title is set to ‘The Counselor – Official Trailer’. There also many other bits of data that we will use, such as ups, downs, num_comments, thumbnail_url and so on. In order to filter out the data that we need, we will use regular expressions. There is one more thing to note: since we are not presenting directories any more but are ready to place content, we have to set the xbmcplugin.setContent to episodes mode.
In the code listed to the left here, we are opening the URL, then – based on regular expression matches – we are discovering the location title, description, date, ups, downs and rating. We are also locating video thumbnails and then passing them on to XBMC. Later in the code, we also try to match the URL to a video provider. With our plug-in we are supporting YouTube, Vimeo and Dailymotion. If this is detected successfully, we call the helper functions to locate the XBMC plugin based playback URL. During this whole parsing process, if any exception is raised, the whole loop is ignored and the next JSON item is parsed.
15
Installing&runningtheadd-on
You can install the add-on using one of the following two methods: • You can copy the plug-in directory to .xbmc/addons. • You can install the plug-in from the zip file. To do so, compress the add-on folder into a zip file using the command:
$ zip -r plugin.video.ludent.zip plugin.video.ludent To install the plug-in from the zip file, open XBMC, go to System then Add-ons, then click ‘Install from zip file’. The benefit of installing from a zip file is that XBMC will automatically try to install all the dependent plug-ins as well. Once you have the plug-in installed, you can run it by going to the Videos Add-ons section of XBMC, selecting Get More… and then clicking on LUD Reddit Viewer. You can access the settings dialog of the plug-in by right-clicking the LUD Reddit Viewer, then selecting ‘Add-on settings’. So, you have seen how robust and powerful XBMC’s extension system is. In this example, we were able to leverage the full power of Python (including those magical regular expression matches) from within XBMC. XBMC itself also offers a robust UI framework, which provides a very professional look for our add-on. As powerful as it may seem, we have only built a video plug-in. XBMC’s extension system also provides a framework for building fully fledged programs (called Programs). We will cover this in a later issue.
61 www.linuxuser.co.uk
Tutorial
Master Vim in easy steps
All the examples presented in this article are tested for Vim version 7.3.547 on Ubuntu 13.04
Just type :help and Vim displays some valuable info for newbies, such as navigation keys and keyboard shortcuts to enable the mouse in xterm or GUI etc
Type :help iccf and Vim displays information (including weblinks) related to donations for the needy in Uganda as well as for the development of the Vim editor
Master Vim in easy steps Take a step ahead, learn some advanced features and get more productive with the Vim editor Advisor
Himanshu Arora is a
software programmer, open source enthusiast and Linux researcher. His articles have been featured on IBM developerWorks, Computerworld and in Linux Journal. He (along with some other Linux freaks) blogs at mylinuxbook.com
Resources
Vim: www.vim.org/download.php
62 www.linuxuser.co.uk
Thanks to a steep learning curve, many people either avoid using Vim or use it on a very basic level. If you’re guilty of either of these then you’re really missing out. Vim, first released in November 1991, is a command-line-based text editor that was developed by Bram Moolenaar as an improved Vi clone for Amiga platform. Besides including all the features of the popular Vi editor, it also contains a lot of new features and so derives its name – from ‘Vi improved’. As well as being a text editor, Vim can be used as an integrated development environment by software programmers. Vim provides a wide range of features, but as it is a command-line utility it has a steep learning curve. Please note that Vim is not a word processor, which means that you cannot do layout-related changes with it. While you will get to know about the real strengths of the Vim editor once you master it,
on the whole some of the prominent pros include fast text editing (because of little to almost no use of the mouse) and easy execution from anywhere on the command line. There are also many GUI front-ends (like gVim) available for Vim. Although it is command-line based, Vim has a loyal following and is extensively used by system administrators and software programmers. Though it is hard to learn and understand all the features that Vim provides, in order to reap its real benefits it is essential to master Vim for your dayto-day work. Through this article you will learn how to work with multiple files, how to indent and fold code, how to search and replace keywords, how to split the Vim window horizontally/vertically, and many other related features provided by Vim. Note: This article assumes that you have some basic knowledge (opening, editing, saving and closing a file) of the Vim editor.
Master Vim in easy steps
Learn some advanced features of Vim
TUTORIal
01
Work with multiple files
While most regular users use Vim to edit one file at a time, it can also be used to edit multiple files in parallel. To open multiple files with the Vim editor, just provide names of all these files as arguments to the vim command. The first file is displayed by default. Type :n to switch to the next file, :prev (or :N) to switch to previous file, :bf to switch to first file, :bl to switch to last file and :bw to close the file.
02
Work with multiple files
Sometimes it is required to open a new or an existing file while editing an already opened set of files. Well, this can be done by typing :e followed by the filename (along with a complete path if it’s not located in the current working directory).
03
Get list of currently opened files in Vim
04
Use wildcard characters
To get the list of files currently opened in the Vim editor, just type :ls. This will display list of opened files along with other information like the current line number of the cursor in a file, file in current active window (represented by %a), file in previous active window (represented by #) etc. Switch between current and previous active windows by typing :b#.
Vim also supports wildcard characters (like *, ? etc) while opening multiple files at a time. This is especially useful in cases where multiple log files begin with a common set of characters like the current date (2013-08-05---.log), or where multiple programming source files have same language extension such as .c or .cpp.
05
Copy-paste across the files
Another frequent requirement is to copy from one file and paste into another. For this, you need to split the Vim window to open both files. This can be done either by using the -o option with the vim command (vim -o ); or by opening a file with vim , then opening the second file in split window mode from within the active Vim window with :split . To switch cursor control between the two windows, press Ctrl+w (a couple of times). Copy-paste operation across the files can be done as follows: open files in split mode (as explained above); copy the required text from one file; press Ctrl+w to switch the control to the other file; then paste the copied text. You can also split a Vim window vertically. All you need to type is :vsp from within an active window.
63 www.linuxuser.co.uk
Tutorial
Master Vim in easy steps
06
Close all files
07
Close files without closing Vim
To close all the opened files in one go, add ‘a’ to regular close options like :q, :wq or :q!. So, to close all the files in a single shot, do something like :qa or :wqa or :qa!. Normally, when the last open file is closed, the Vim application also exits. But, if it is required to close the last open file without closing the application, then just type :bd. This closes the current file and opens an empty buffer.
While most regular users use Vim to edit one file at a time, it can also be used to edit multiple files in parallel
08
Fold blocks of code
Another frequent requirement – which software programmers usually have – is to fold some blocks of code in order to save window space. This helps them to focus on the required code blocks. Vim does support block folds. For this, the first step is to bring cursor inside the
64 www.linuxuser.co.uk
braces of the function definition. For example, bring the cursor to the first line of the function body. Once there, just type va}zf. As you type this, you will observe that va is used to activate visual mode in Vim, while } selects the complete block including curly braces and zf enables the fold. To open the fold, press zo; to close it, press zc. Make sure that the cursor is within the block while performing fold open and close. Typing va}zf creates a fold (that is invisible). zo and zc work only when a fold is created. An existing fold can be deleted by typing zd.
09
Manual indenting
Another important feature that Vim provides is code indentation. Software programmers are required to indent their code and Vim provides easy steps for code indentation. To indent a single line of code, you can use >> or > or
Master Vim in easy steps
Learn some advanced features of Vim
TUTORIal
For those programmers who want everything automated, Vim also provides an auto-indenting feature
[number-of-lines] or 10> Master the RasPi camera >> LTE Open Phone reviewed The world’s favourite version control >> Visualise data with Graphviz >> Protect your network with FOSS system isn’t as easy as it could be
Distributed in the Rest of the World by Marketforce, Blue Fin Building, 110 Southwark Street, London, SE1 0SU. ☎ 0203 148 8105
Disclaimer
The publisher cannot accept responsibility for any unsolicited material lost or damaged in the post. All text and layout is the copyright of Imagine Publishing Ltd. Nothing in this magazine may be reproduced in whole or part without the written permission of the publisher. All copyrights are recognised and used specifically for the purpose of criticism and review. Although the magazine has endeavoured to ensure all information is correct at time of print, prices and availability may change. This magazine is fully independent and not affiliated in any way with the companies mentioned herein. If you submit material to Imagine Publishing via post, email, social network or any other means, you automatically grant Imagine Publishing an irrevocable, perpetual, royalty-free license to use the images across its entire portfolio, in print, online and digital, and to deliver the images to existing and future clients, including but not limited to international licensees for reproduction in international, licensed editions of Imagine products. Any material you submit is sent at your risk and, although every care is taken, neither Imagine Publishing nor its employees, agents or subcontractors shall be liable for the loss or damage. © Imagine Publishing Ltd 2013 ISSN 2041-3270m
98 www.linuxuser.co.uk
CONVERT PRINT CREATE COMBINE & MODIFY 100% Standalone - No Office Automation
many MORE
SCAN FOR
20% SAVINGS
