Parallel and Distributed Processing

Course Information

  • Batch: 72th
  • Course Code: CSE-411
  • Credits: 03
  • Course Teacher: Md Habibul Basar Faruq
  • Email: mh.faruq06@gmail.com
  • Google Classroom: Join

Course Description

This course introduces the principles and practices of parallel and distributed computing. Topics include models of parallelism, architectures, communication and synchronization, performance metrics, parallel algorithms, and distributed system fundamentals. Students will gain hands-on experience with frameworks such as MPI, OpenMP, Hadoop, and Spark, applying them to real-world problem solving.

Weekly Topics and Resources

  • Week 1: Introduction to Parallel and Distributed Systems
    Motivation, real-world applications, system architectures. | Lecture slide 01

  • Week 2: Models of Parallelism
    Flynn’s taxonomy (SISD, SIMD, MISD, MIMD), shared vs distributed memory. | Lecture Note 02

  • Week 3: Performance Metrics
    Speedup, efficiency, scalability, Amdahl’s Law, Gustafson’s Law. | Lecture slide 03

  • Week 4: Shared Memory Programming
    Threads, synchronization, OpenMP basics. | Lecture Slide

  • Week 5: Message Passing Programming
    MPI basics: point-to-point and collective communication.

  • Week 6: Synchronization & Coordination
    Barriers, locks, deadlocks, load balancing.

  • Week 7: Parallel Algorithms
    Sorting, matrix multiplication, graph algorithms.

  • Week 8: Distributed Systems Basics
    Client-server, RPC, socket programming.

  • Week 9: Distributed Coordination
    Consensus algorithms, clock synchronization, CAP theorem.

  • Week 10: Big Data & Frameworks
    MapReduce paradigm, Hadoop, Spark.

  • Week 11: GPU and Accelerator Computing
    CUDA and OpenCL basics for data-parallel processing.

  • Week 12: Project Presentations & Future Trends
    Cloud, edge, and quantum computing; student project demos.

Course Credits

  • Lecture Slides & Materials: Instructor’s prepared content, research papers, and online tutorials.
  • Instructor: Md Habibul Basar Faruq
  • Contact: mh.faruq06@gmail.com

Suggested Textbooks & References

  • Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar – Introduction to Parallel Computing, 2nd Ed., Addison Wesley.
  • George Coulouris, Jean Dollimore, Tim Kindberg – Distributed Systems: Concepts and Design.
  • Michael J. Quinn – Parallel Programming in C with MPI and OpenMP.
  • Kai Hwang, Jack Dongarra, Geoffrey C. Fox – Distributed and Cloud Computing: From Parallel Processing to the Internet of Things.

Assessment & Evaluation

  • Assignments & Quizzes –
  • Midterm Exam –
  • Lab/Practical –
  • Project (Team-based) –
  • Final Exam –