Counting sort(Non Comparison Sort)

Comparison Sorting
 A comparison sort is a type of sorting algorithm that only
reads the list elements through a single comparison operation
(often a "less than or equal to" operator or a three-way
comparison) and determines which of two elements should
occur first in the final sorted list.
 What is common to all these algorithms?

– Make comparisons between input elements like:

ai < aj, ai ≤ aj, ai = aj,

ai ≥ aj, or

ai > aj

2

Examples of Comparison Sorting
1. Bubble sort
2. Insertion sort
3. Selection sort
4. Quick sort
5. Heap sort
6. Merge sort
7. Odd-even sort
8. Cocktail sort
9. Cycle sort
10. Merge insertion (Ford-Johnson) sort
11. Smoothsort
12. Timsort
3

Limitations of Comparison Sorting
 There are fundamental limits on the performance of
comparison sorts.
 To sort n elements, comparison sorts must make (n log n)
comparisons in the worst case.
 That is a comparison sort must have a lower bound of
Ω(n log n) comparison operations, which is known as linear or
linearithmic time.
 This is a consequence of the limited information available
through comparisons alone

4

Why does it takes so much of time?
 A decision tree is used to represent the comparisons of a
sorting algorithm. Assume that all inputs are distinct. A
decision tree compares all possible inputs to each other to
determine the sequence of outputs.
 Decision Tree for three numbers a1, a2, a3 : If at the root, a1 ≤
a2 gο left and compare a2 to a3, otherwise go to right and
compare a1 to a3. Each path represents a different ordering of
a1, a2, a3.

 This type of decision tree will have n! leaves.
 Any comparison based sorting algorithm will have to go
through the steps in the decision tree as a minimum

 So for n inputs, the tree must have n! leaves. A binary tree of
height h has no more than 2h leaves.
 Now n ! <= 2h
 Taking log on both side: lg(n !) <= h
 According to stirling’s approximation we have:

 So it can be written h= Ω(n log n). This means we need to do at
least n log n comparisons to reach the bottom of the tree

How Fast Can We Sort?
• Selection Sort, Bubble Sort, Insertion Sort, Cocktail
sort, Cycle sort: O(n2)
• Smooth sort, Odd-even sort, Timsort: O(n)
• Heap Sort, Merge sort: O(n log n)
• Quicksort:
– Average: O(n log n)
– Best: O(n log n) (simple partition) or O(n) (three-way
partition and equal keys)
8

Non Comparison Sorting
 There are some sorting algorithms that perform sorting without
comparing the elements rather by making certain assumptions
about the data. They are called the non comparison sorting.
 Non comparison sorting include:
1. Counting sort (indexes using key values)
2. Radix sort (examines individual bits of keys)
3. Bucket sort (examines bits of keys)
 These are Linear sorting algorithms. Linear sorts are NOT
“comparison sorts”.
 They make certain assumptions about the data. These type of
sorting algorithm does not need to go through the comparison
decision tree.
9

Counting sort
 Counting sort assumes that each of the n input elements is an
integer in the range 0 to k. that is n is the number of elements and
k is the highest value element.
 Consider the input set : 4, 1, 3, 4, 3. Then n=5 and k=4
 Counting sort determines for each input element x, the number of
elements less than x. And it uses this information to place
element x directly into its position in the output array. For
example if there exits 17 elements less that x then x is placed into
the 18th position into the output array.
 The algorithm uses three array:
Input Array: A[1..n] store input data where A[j] {1, 2, 3, …, k}
Output Array: B[1..n] finally store the sorted data
Temporary Array: C[1..k] store data temporarily

Counting Sort
1. Counting-Sort(A, B, k)
2. Let C[0…..k] be a new array
3. for i=0 to k
4.
C[i]= 0;
5. for j=1 to A.length or n
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
9. for j=n or A.length down to 1
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;

Counting Sort
3. for i=0 to k
4.
C[i]= 0;
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;

[Loop 1]
[Loop 2]

[Loop 3]
[Loop 4]

Counting-sort example
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

C:
1

B:

2

3

4

5

6

7

8

Executing Loop 1
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

0

0

0

0

C: 0
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

1

0

0

0

C: 0
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

1

0

0

1

C: 0
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

1

1

0

1

C: 0
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

1

1

0

1

C: 1
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

1

0

1

C: 1
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

2

0

1

C: 1
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

2

0

1

C: 2
1

B:

2

3

4

5

6

7

8

Executing Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

3

0

1

C: 2
1

B:

2

3

4

5

6

7

8

End of Loop 2
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

3

0

1

C: 2
1

B:

2

3

4

5

6

7

8

Executing Loop 3
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

3

0

1

C: 2
0

C: 2
1

B:

1

2

3

4

5

2

2

3

0

1

2

3

4

5

6

7

8

Executing Loop 3
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

3

0

1

2

C: 2

2

0

1

2

3

4

5

C: 2

2

4

3

0

1

1

B:

2

3

4

5

6

7

8

Executing Loop 3
1

2

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

1

2

3

4

5

C: 2

2

4

3

0

1

0

1

2

3

4

5

2

4

7

0

1

A:

C: 2
1

B:

2

3

4

5

6

7

8

Executing Loop 3
1

2

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

1

2

3

4

5

C: 2

2

4

7

0

1

0

1

2

3

4

5

2

4

7

A:

C: 2
1

B:

2

3

4

1

7
5

6

7

8

Executing Loop 3
1

2

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

1

2

3

4

5

C: 2

2

4

7

7

1

0

1

2

3

4

5

2

4

7

7

A:

C: 2
1

B:

2

3

4

5

8
6

7

8

End of Loop 3
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

2

4

7

7

8

C: 2
1

B:

2

3

4

5

6

7

8

Executing Loop 4
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

2

4

7

7

8

C: 2
1

B:

2

3

4

5

6

7

8

Executing Loop 4
1

A:

2

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

C: 2
1

B:

1

2

3

4

5

2

4

7

7

8

2

3

4

5

6

7

J=8, then A[ j ]=A[8]=3
And B[ C[ A[j] ] ]
=B[ C[ 3 ] ]
=B[ 7]
So B[ C[ A[j] ] ] ←A[ j ]
=B[7]←3

8

Executing Loop 4
1

A:

2

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

C:

1

2

3

4

5

2

2

4

6

7

8

1

B:

2

3

4

5

6

7

3

J=8, then A[ j ]=A[8]=3
Then C[ A[j] ]
= C[ 3 ]
=7
So C[ A[j] ] = C[ A[j] ] -1
=7-1=6

8

Executing Loop 4
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

2

4

6

7

8

C: 1
1

B:

2

0

3

4

5

6

7

3

8

Executing Loop 4
1

C:

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

1

2

4

5

7

8

1

B:

2

0

3

4

5

6

7

3

3

8

Executing Loop 4
1

C:

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

1

2

3

5

7

8

1

B:

2

0

3

4

2

5

6

7

3

3

8

Executing Loop 4
1

C:

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

0

2

3

5

7

8

1

B: 0

2

0

3

4

2

5

6

7

3

3

8

Executing Loop 4
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

2

3

4

7

8

C: 0
1

B: 0

2

0

3

4

5

6

7

2

3

3

3

8

Executing Loop 4
1

3

4

5

6

7

8

2

5

3

0

2

3

0

3

0

A:

2

1

2

3

4

5

2

3

4

7

7

C: 0
1

B: 0

2

0

3

4

5

6

7

8

2

3

3

3

5

End of Loop 4
1

2

3

4

5

6

7

8

A: 2

5

3

0

2

3

0

3

1

2

3

4

5

2

2

4

7

7

0

C: 0
1

B: 0

2

3

4

5

6

7

8

0

2

2

3

3

3

5

Sorted data in Array B

Time Complexity Analysis
3. for i=0 to k
4.
C[i]= 0;
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;

[Loop 1]

Loop 1 and 3
takes O(k) time

[Loop 2]

[Loop 3]
[Loop 4]
Loop 2 and 4
takes O(n) time

Time Complexity Analysis
• So the counting sort takes a total time of: O(n + k)
• Counting sort is called stable sort.
– A sorting algorithm is stable when numbers with
the same values appear in the output array in the
same order as they do in the input array.

Counting Sort Review
• Why don’t we always use counting sort?
– Depends on range k of elements.
• Could we use counting sort to sort 32 bit integers? Why or
why not?

Counting Sort Review
• Assumption: input taken from small set of numbers of size k
• Basic idea:
– Count number of elements less than you for each element.
– This gives the position of that number – similar to selection
sort.
• Pro’s:
– Fast
– Asymptotically fast - O(n+k)
– Simple to code
• Con’s:
– Doesn’t sort in place.
– Requires O(n+k) extra storage.

MD. Shakhawat Hossain
Student of Computer Science & Engineering Dept.
University of Rajshahi

Counting sort(Non Comparison Sort)

In this document

More Related Content

What's hot

Viewers also liked

Similar to Counting sort(Non Comparison Sort)

More from Hossain Md Shakhawat

Recently uploaded

Counting sort(Non Comparison Sort)