Glossário - Definições importantes para o curso

Fundamental concepts

Distributed algorithms:
A distributed algorithm is an algorithm designed to run on computer hardware constructed from interconnected processors. Distributed algorithms are used in many varied application areas of distributed computing, such as telecommunications, scientific computing, distributed information processing, and real-time process control. Standard problems solved by distributed algorithms include leader election, consensus, distributed search, spanning tree generation, mutual exclusion, and resource allocation. (reference Wikipedia)
Distributed computing:
Distributed computing refers to the use of distributed systems to solve computational problems. In distributed computing, a problem is divided into many tasks, each of which is solved by one processor. (adapted from Wikipedia)
Fine grain paralellism:
It's a class of paralellism which the grains represents that individual tasks are relatively small in terms of code size and execution time. The finer the granularity, the greater the potential for parallelism and hence speed-up, but the greater the overheads of synchronization and communication. In order to attain the best parallel performance, the best balance between load and communication overhead needs to be found. If the granularity is too fine, the performance can suffer from the increased communication overhead. On the other side, if the granularity is too coarse, the performance can suffer from load imbalance. Coarse-grained is the opposite: data are communicated infrequently, after larger amounts of computation.
Graphics Processing Unit; A processing unit dedicated to floating-point for graphics processing. E.g., NVIDIA video cards like GeForce or Quadro have multiple GPUs each.
GPGPU/GPU Computing:
General-Purpose computation on Graphics Processing Units, also known as GPU Computing. It is the use of GPUs with high-performance many-core processors to accelerate a wide range of applications ( ). For NVidia, GPGPU required using graphics programming languages like OpenGL and Cg to program the GPU while GPU computing is the correct term to general purpose scientific and engineering computing using graphics processing unit.
Granularity is a measure of the size of the components, or descriptions of components, that make up a system. Granularity is the relative size, scale, level of detail or depth of penetration that characterizes an object or activity. It is the extent to which a system is broken down into small parts or grains, either the system itself or its description or observation. In parallel computing, granularity means the ratio of computation to the amount of communication.
Multiple Instructions, Multiple Data. This is the case where many cpus process different data assinchronously. The memory can be shared or not. This is commonly called distributed computing (different computers in a network) or multi-core (many cpus in a same silicon chip)
Chip with two or more independent cores (processors)
Row major:
Row major is the way CUDA, C and other programming languages allocate arrays in the system's RAM. They allocate each consecutive element of a row in consecutive memory positions, and each successive row immediately following its predecessor. Other programming languages such as Fortran use column major schemes.

Single Instruction, Multiple Data.

In this processing paradigm, the architecture ensures that every thread is running exactly the same instruction of the program as its concurrent threads, though on different data. i.e., threads are inherently synchronized.