Parallel/Concurrent Languages:

A concurrent language is defined as one which uses the concept of simultaneously executing processes or threads of execution as a means of structuring a program. A parallel language is able to express programs that are executable on more than one processor. Both types are listed as concurrency is a useful tool in expressing parallelism, but it is not necessary. In both cases, the features must be part of the language syntax and not an extension such as a library.

Picking a languages to use in order to write Parallel Programs includes a deeper thought, and a couple on initial questions. For example, different programs hide the machine in different ways. Hiding improves programmability (productivity), portability, while exposing gives programmers control to improve performance.

There are two components to picking a language- control and data (communication/sharing). What is the parallel control model?

What is the model for sharing/communication?

Programming Models:

Message passing. Message passing is probably the most widely used parallel programming model today. Message-passing programs create multiple tasks, with each task encapsulating local data. Each task is identified by a unique name, and tasks interact by sending and receiving messages to and from named tasks. In this respect, message passing is really just a minor variation on the task/channel model, differing only in the mechanism used for data transfer. For example, rather than sending a message on ``channel ch,'' we may send a message to ``task 17.'' Most message-passing systems create a fixed number of identical tasks at program startup and do not allow tasks to be created or destroyed during program execution. These systems are said to implement a single program multiple data (SPMD) programming model because each task executes the same program but operates on different data.

Data Parallelism. Calls for exploitation of the concurrency that derives from the application of the same operation to multiple elements of a data structure, for example, ``add 2 to all elements of this array,'' or ``increase the salary of all employees with 5 years service.'' A data-parallel program consists of a sequence of such operations. As each operation on each data element can be thought of as an independent task, the natural granularity of a data-parallel computation is small, and the concept of ``locality'' does not arise naturally. Hence, data-parallel compilers often require the programmer to provide information about how data are to be distributed over processors, in other words, how data are to be partitioned into tasks. The compiler can then translate the data-parallel program into an SPMD formulation, thereby generating communication code automatically.

Shared Memory. In the shared-memory programming model, tasks share a common address space, which they read and write asynchronously. Various mechanisms such as locks and semaphores may be used to control access to the shared memory. An advantage of this model from the programmer's point of view is that the notion of data ``ownership'' is lacking, and hence there is no need to specify explicitly the communication of data from producers to consumers. This model can simplify program development. However, understanding and managing locality becomes more difficult, an important consideration on most shared-memory architectures. It can also be more difficult to write deterministic programs.

When	Who/Where	Description
		ABCPL	An object-Based Concurrent Language. Model of concurrency based on parallel active objects.
1983	DoD	ADA	The Ada language of DoD fame
		Adl	Data parallel functional programming languages for distributed memory architectures.
		Adsmith	Object-Based DSM Environment on PVM.
		Alewife	Large-scale multiprocessor with shared memory and message passing.
1990	NYU	ALLOY	support functional, object oriented and logic programming styles in a unified and controlled framework.
		AM	Active Messages. Simple primitives exposing full hardware performance to higher communication layers.
		AMDC	Active-Message Driven Computing. Computational model and system based on computation at abstract locations, and built on active messages.
		APAR	Parallel Architectures group. Portable parallel programming environment, parallel 3D terrain analysis and visualization.
		AppLeS	Application Level Scheduling. Scheduling agents for resource control on production heterogeneous systems.
		Automap	Task parallel coordination language for dynamic applications.
1987		BLAZE	Language for shared-memory using data distribution descriptions
		BSP	Bulk Synchronous Parallel model research group.
1994	U. Wisc	C**	extends C++ for large-grain data parallelism; syntax defining aggregate operations, reductions, and subset selection (slices).
		C4	Canonical Classes for Concurrency Control. C++ classes for parallel programming.
1992	Cal Tech	CC++	parallel programs to be developed from simpler components using sequential, parallel, and concurrent composition.
		Chant	A talking threads package -- lightweight threads communicating between processors.
1995	UIUC	Charm	A Portable Concurrent Object Oriented System based on C++
		Cid	Parallel extension of C that provides a programming model of MIMD threads plus shared objects.
1998	MIT	Cilk	Efficient execution of multithreaded computations.
1998	SGI	Co-Array Fortran	explicit notation for data decomposition, such as that often used in message-passing models; decomp to be added to F95
1996	UT	CODE	abstract declarative graphical environment for parallel programming
		Code	Visual parallel programming system.
1993	UIUC	Concurrent Aggregates language	dynamically-typed concurrent object-oriented language for programming large-scale parallel machines;a programming model similar to Actors
		Concurrent Pascal
		Concurrent Prolog
		Craft	Cray's parallel programming language ("better than HPF")
		CRL	C Region Library. All-software distributed shared memory system.
1978	Oxford	CSP	(Communicating Sequential Processes) language for describing patterns of interaction. It is supported by an elegant, mathematical theory, a set of proof tools, and an extensive literature
1996	U. Parma	CUBL	concurrent object language
1994	Dartmouth	Dapple	C++ class library designed to provide the illusion of a data-parallel programming language on conventional hardware
		DDD	Dynamic Distributed Data. Parallel programming model based on dynamic graph-like datastructures.
1985	Eiffel Software	Eiffel	OO-based parallelism; one of the first
1995	Erlang	Erlang	functional programming, programmer-explicit messages, transparent distribution
		Excalibur	Compiling array-oriented languages onto parallel machines.
		Express
1984	ICOT	FGHC	(Flat Guarded Horn Clauses) a concurrent logic language, that supports committed-choice non-determinism and stream communication
		FM	Fortran M. Task-parallel extensions to Fortran; integration of task and data parallelism.
		Fork95	C-based parallel programming language, targetted at SB-PRAM.
1995		Fortran 95	introduced FORALL, parallel do-loop; and INDEPENDENT directive for DO loops
1991	Rice, CT	Fortran D	Fox & Kennedy: a language extension for distributed-memory systems. This is later one of the parents of High Performance FORTRAN (HPF).
1978	IBM	FP	John Backus’ original functional programming language (some claim it was penance for Fortran)
		Fx	Parallelizing Fortran compilers and applications.
		GA	Global Arrays. Shared-memory programming interface for distributed-memory computers.
1993	NRAO	Glish	C++ library for distributed applications
		GLU	Granular Lucid. Programming system for constructing parallel and distributed applications.
1990	U. Glasgow	Haskell	polymorphicly typed, lazy, purely functional language based on Lambda calculus
1999	UIUC	HPC++	A standard model for parallel programming using C++.
1993	Rice	HPF	High Performance Fortran. Data-parallel and data distribution extensions to Fortran.
1996	UIUC	ICC++	object-oriented programming style (fine-grained data structures and concurrency) which supports a wide variety of irregular and regular parallel applications.
1978	UCIrvine	Id	Arvind's original Dataflow language (originally "Irvine Dataflow"), moved to MIT.
1996	Erlangen	JavaParty	extends the capabilities of Java to distributed computing environments.
1997	WoTUG	JavaPP	introduces the CSP/occam model into Java threads
1976	ONERA	LAU	Static dataflow language for the LAU machine
1995	Lucent	Limbo	a programming language intended for applications running distributed systems on small computers
		Linda - Ada-linda	Linda in Ada
		Linda - Glenda	Linda on PVM
		Linda - Javelin	Distributed Linda Using Java
1985	Yale	Linda Original	The original language of get & put tuples and operating on them
		Linda - POSYBL	public-domain implementation of the Linda parallel processing environment,
1977		Lucid	functional language making iteration "mathemetically respectable"; became dataflow language
	U VA	Mentat	Object-oriented parallel language based on C++
		Modula-2*	High-level explicitly parallel imperative programming language.
	RAS	mpC	A parallel superset of C for programming distributed memory machines built on MPI
		MPC++	Massively Parallel, Message Passing, Meta-level Processing C++ designed to exploit multithreading and message-driven execution on RWC-1 MPP .
2001	U AZ	MPD	C extension supporting all three of multithreaded, parallel, distributed programming.
1980	MIT	Multilisp	A version of the Lisp dialect Scheme extended with constructs for parallel execution.
	CMU	NESL	SCANDAL - Super Compilers and Advanced Languages. Nested data parallel languages (NESL), efficient parallel algorithms.
1984	Oxford	Occam	The canonical language expession of Hoare's CSP
		OOF90	High performance object-oriented programming in Fortran 90.
		Orca	Parallel programming language for distributed systems, based on shared data-object model.
2004	U.Edinb.	P#	Concurrent Prolog for .NET
		P3L	Skeleton-based parallel programming language with template-based implementation.
		Para++	Portable C++ bindings for message passing libraries.
	U. Stuttgart	Parallaxis-III	A Structured Data-Parallel Programming Language
1995	UT	Parlance	Lisp extension supporting medium-scale parallelism on SMPs.
1986	Imp. Coll.	Parlog	An AND-parallel Prolog, with guards and committed choice nondeterminism
	IN	pC++/Sage++	Portable parallel C++ for high performance computers.
		PET	Message passing, parallel I/O, checkpointing, and run-time tools and monitoring.
		PETSc	Portable, Extensible, Toolkit for Scientific Computation. Parallel software for the numerical solution of PDEs.
		pH	A parallel, eagerly-evaluated version of Haskell.
1987	UW	PRESTO	C++ based environment for writing object-oriented parallel programs for shared-memory multiprocessors.
		Proteus	Prototyping language and environment for high-level parallel programs.
1996	UCB / ICSI	pSather	extends Sather (OOL) with threads, synchronization and data distribution
		PSI	Parallel algebraic reductions on array expressions based on psi calculus.
1989	ORNL	PVM	Parallel Virtual Machine. Programming a heterogeneous network of machines as a single distributed memory parallel machine.
1998	EPFL	sC++	C++ with a very few new keywords defining active objects and synchronization primitives
	Delftware	Scriptic	expression based extension to Java, targeted at shared memory systems; extras mainly founded on the theory of Process Algebra;
1983	LLNL	Sisal	Dataflow / functional language
		Split-C	Parallel extension to C with global address space for distributed-memory multiprocessors.
		SR	Synchronizing Resources. Concurrent programming language.
1997	MIT	Star-P Parallel MATLAB	MATLAB modified to exploit parallelism in its output evaluation forms; now at Interactive Supercomputing of Waltham,MA
		Synergy	Parallel programming using passive object flow.
	NEU	TOP-C	Task-oriented Parallel C.
		UC	A set-based data-parallel programming language.
		uC++	Light-weight concurrency in C++.
1999	UCB/LBNL	UPC	Unified Parallel C
		V	Nested data-parallel extension of C.
1979	MIT	VAL	Value-oriented Algorithmic Language; a dataflow language; iteration and error handling, lacking in recursion and I/O
1996	ZH Micro	Z++	Object-oriented extension of C++ to multi-threading and distributed computing.
		ZPL	Portable data-parallel array language for scientific and engineering computation.