Lesson about delegation from OpenCL

Using OpenCL and the complexities / benefits that comes with it reminds me of an issue of trade-off in delegation. You can always do things the simple straightforward, serial way but there is a limit of how much and how fast it can finish the tasks. We can scale things better by utilizing more external agents and manage it. However using additional agents means delegating tasks to those agents and manage the overhead and unpredictability that comes with it. This is why the code to achieve the same thing using OpenCL can be quite complex since you need to manage the data and logic that comes to and from those computing units and get them to work together properly.

It's not just cost that is increased though, the risk is also higher. When you delegate, you add another factor with its own set of quirkiness to your system. Each agent you add, you add another risk of it fail in some way and effect the overall system. In OpenCL, certain GPU might not behave as it should, your calculation might fail in certain configuration. So, you will need to add more activities to manage the risk. How it can be anticipated, localized and how the system can cope gracefully when it happen.

So, since delegation is not free it is important that the scale of the problem would need to justify the use of delegation.  You can add more things, method, people to your system, but when it does not fit your scale, it will become a burden since your output, what you get, is lower than what you need to expense. In term of OpenCL, the low complexity or low number of calculation probably not worth the trouble. For example, even when you ding the formula is complex and can be paralellize but if your expected data to be processd is not much, it probably better to just code it the traditional way.

Delegation can be costly and risky, but if you need your solution to scale better, the return can be very much worth it.

Translating Traditional Program Logic to OpenCL

If you just started to look at OpenCL and see the example of the code, it can be quite confusing on how it got that way from its traditional program logic counterpart. Take a look at an example of comparison below taken from PDF Overview on OpenCL site :

You can see that the calculation is intact but you can't seem to find where the for loop counterpart is.

It is because, the logic has been separated between host (your main program) and kernel (that will be sent to device). So, you need to look at both of them to get the whole picture. There is some data partitioning done that you can take a look at the code at the host. Together with the batch processing happen in kernel code you can start to see the implied loop there.

Due to its parallelism and batch processing nature, the logic in OpenCL can look more declarative and more separated compare to traditional one. So, the logic that you expect to find will probably be implied somewhere in host and kernel code. This could help if you try to read the existing code.

Getting Started with OpenCL

I started working with OpenCL recently on the project at work. It's quite exhausting to learn initially since OpenCL has quite steep learning curve. However, once you have started to picked up some of the core concept, it could be quite interesting. Also, you will add another valuable tool to your programming arsenal, especially in the aspect of performance and speedup. You will start to be able to exploit not just CPU but also any other computing device that exist on the system like GPU quite conveniently.

Here's summary and pointers of how to get started with it from what I note so far.

Overview

You can get an initial overview from the surces below :

• OpenCL Overview (PDF) from its official site is quite easy to skim. Just skim it and don't bother with the details like code, it will be clearer soon
• Also, skimming the Official Specification and Quick Reference Card could help too to get an initial picture of what it is

 As for its Wikipedia Page, I don't find it really informative at the moment, at least if you want to get an overview of what OpenCL is and its relevance.

As usual, google search and some blog entries would not hurt too. However, it's better not to take too much time researching things online since I think your time is better spend reading the book below.

Book : OpenCL in Action

There is not many technical book that you can enjoy reading cover to cover. This one is one of them. It's better you read from beginning to end serially. It build up from chapter to chapter. Also, it does not cross reference things too much between materials which could be annoying on some technical books. The materials coverage is very good, from the history, introduction to advanced topics.

If you have some questions on your mind from reading other sources previously, there is a good chance that you will start to piece things together when reading this book. It gives a good foundation for understanding why OpenCL is the way it is. OpenCL syntax and functions will still be quite overwhelming even after you finished with the book but knowing the concept behind will make it much more manageable.

I find the highlights of the book is its use of analogies that helps a lot in explaining the abstract concept and architecture of OpenCL.

Revisit Specification and Reference Card

If you have skimmed the specification and reference card above, it's time to revisit it again. You will now find it much more clearer now and they are starting to be more useful. It will also add more details from what is covered on the book. Personally, I really like the layout on reference card which help in getting a good high level picture of different parts of OpenCL with it's colorful boxes and groupings.

At this point, you can start to work on your specific issues and continue learning from there. As usual, after the initial learning, the actual work is where we will learn most. This is especially true in complex technology like OpenCL.