الاحتماليه

Communication II

Topics

1-    Information theory
2-    Coding of discrete Sources
3-    Channel Coding
4-    Introduction to Digital Signal processing (DSP)
5-    Digital Filter Design
6-    Selected Communication Systems
a)    Optical communication systems
b)    Satellite communication systems
c)    Spread spectrum systems
d)    Global  System for Mobile communication (GSM)




References

1)    G. Stremler , Introduction to Communication Systems(Textbook).
2)    A. Glover , Digital Communication.
3)    Introductory  to Digital Signal Processing.
4)    Understanding-Digital-Signal-Processing



Overview: What is Information Theory?

Key idea: The movements and transformations of information, just like
those of a fluid, are constrained by mathematical and physical laws.
These laws have deep connections with:
•    probability theory, statistics
•    thermodynamics (statistical physics)
•    spectral analysis, Fourier (and other) transforms
•    sampling theory, prediction, estimation theory
•    electrical engineering (bandwidth; signal-to-noise ratio)
•    complexity theory (minimal description length)
•    signal processing, representation, compressibility

Two fundamental questions of communication theory:

1. What is the ultimate data compression?
(answer: the entropy of the data, H, is its compression limit.)

2. What is the ultimate transmission rate of communication?
(answer: the channel capacity, C, is its rate limit.)

All communication schemes lie in between these two limits on the compressibility of data and the capacity of a channel. Information theory
can suggest means to achieve these theoretical limits. But the subject
also extends far beyond communication theory.




Mathematical Foundations; Probability Rules;
Bayes Theorem

What are random variables? What is probability?

We deal with previous study with waveforms whose values can be specified exactly  are said to be “ deterministic”. Now we consider a method for describing a nondeterministic , or “random”  waveform signal. Random variables are variables that take on values determined by probability distributions. They may be discrete or continuous, in either their domain or their range. For example, a stream of ASCII encoded text characters in a transmitted message is a discrete random variable, with a known probability distribution for any given natural language. An analog speech signal represented by a voltage or sound pressure wave form as a function of time (perhaps with added noise), is a continuous random variable having a continuous probability density function


Probability is the measured (numerically ) the favorable outcomes of a given experiment.

Suppose that some experiment  is considered in which the outcome dose not remain constant . on of the possible outcome of the experiment is label  A. Tossing  a coin is such an experiment , the possible outcomes being  head and  tails. If the experiment is repeated  N times , suppose that the outcome  A will occur  NA times .