Your Progress
60% complete. Continue learning to master DSA!
Heap Sort
Heap sort is an efficient comparison-based sorting algorithm that uses a binary heap data structure to sort elements in an array with guaranteed O(n log n) time complexity.
Key Takeaways
- Heap sort uses a binary heap data structure to efficiently sort elements
- Time complexity is O(n log n) for all cases (best, average, worst)
- Space complexity is O(1) as it sorts in-place
- It's not a stable sorting algorithm but offers consistent performance
- Heap sort has better worst-case performance than quicksort but is often slower in practice
Understanding Heap Sort
Heap sort is a comparison-based sorting algorithm that uses a binary heap data structure. The algorithm divides the input into a sorted region and an unsorted region, and iteratively shrinks the unsorted region by extracting the largest element from it and inserting it into the sorted region.
How Binary Heaps Work
A binary heap is a complete binary tree where each node satisfies the heap property:
- Max Heap: Each parent node is greater than or equal to its children
- Min Heap: Each parent node is less than or equal to its children
Heap sort typically uses a max heap to sort elements in ascending order. In a binary heap represented as an array:
- For a node at index i, its left child is at index 2i + 1
- Its right child is at index 2i + 2
- Its parent is at index floor((i-1)/2)
Algorithm Steps
- Build a max heap from the input array
- Swap the root (maximum element) with the last element of the heap
- Reduce the heap size by 1 (exclude the last element)
- Heapify the root to maintain the max heap property
- Repeat steps 2-4 until the heap size becomes 1
Visual Example
Let's sort the array [4, 10, 3, 5, 1] using heap sort:
Original array: [4, 10, 3, 5, 1] Step 1: Build a max heap - First, we represent the array as a complete binary tree: 4 / \ 10 3 / \ 5 1 - Heapify the tree starting from the last non-leaf node: 4 10 10 / \ / \ / \ 10 3 → 4 3 → 5 3 / \ / \ / \ 5 1 5 1 4 1 Final Max Heap: [10, 5, 3, 4, 1] Step 2: Sort the array - Swap root (10) with last element (1): [1, 5, 3, 4, 10] - Reduce heap size to 4 and heapify: [5, 4, 3, 1, 10] - Swap root (5) with last element (1): [1, 4, 3, 5, 10] - Reduce heap size to 3 and heapify: [4, 1, 3, 5, 10] - Swap root (4) with last element (3): [3, 1, 4, 5, 10] - Reduce heap size to 2 and heapify: [3, 1, 4, 5, 10] - Swap root (3) with last element (1): [1, 3, 4, 5, 10] - Reduce heap size to 1 (only one element left) Final sorted array: [1, 3, 4, 5, 10]
Implementing Heap Sort
JavaScript Implementation
function heapSort(arr) { const n = arr.length; // Build max heap for (let i = Math.floor(n / 2) - 1; i >= 0; i--) { heapify(arr, n, i); } // Extract elements from heap one by one for (let i = n - 1; i > 0; i--) { // Move current root to end [arr[0], arr[i]] = [arr[i], arr[0]]; // Call heapify on the reduced heap heapify(arr, i, 0); } return arr; } // Heapify a subtree rooted at node i function heapify(arr, n, i) { let largest = i; // Initialize largest as root const left = 2 * i + 1; // Left child const right = 2 * i + 2; // Right child // If left child is larger than root if (left < n && arr[left] > arr[largest]) { largest = left; } // If right child is larger than largest so far if (right < n && arr[right] > arr[largest]) { largest = right; } // If largest is not root if (largest !== i) { // Swap and continue heapifying [arr[i], arr[largest]] = [arr[largest], arr[i]]; heapify(arr, n, largest); } } // Example usage const array = [4, 10, 3, 5, 1]; console.log(heapSort(array)); // Output: [1, 3, 4, 5, 10]
Python Implementation
def heap_sort(arr): n = len(arr) # Build a max heap for i in range(n // 2 - 1, -1, -1): heapify(arr, n, i) # Extract elements one by one for i in range(n - 1, 0, -1): # Move current root to end arr[i], arr[0] = arr[0], arr[i] # Call heapify on the reduced heap heapify(arr, i, 0) return arr def heapify(arr, n, i): largest = i # Initialize largest as root left = 2 * i + 1 # Left child right = 2 * i + 2 # Right child # If left child exists and is greater than root if left < n and arr[left] > arr[largest]: largest = left # If right child exists and is greater than largest so far if right < n and arr[right] > arr[largest]: largest = right # If largest is not root if largest != i: # Swap arr[i], arr[largest] = arr[largest], arr[i] # Recursively heapify the affected sub-tree heapify(arr, n, largest) # Example usage array = [4, 10, 3, 5, 1] print(heap_sort(array)) # Output: [1, 3, 4, 5, 10]
C++ Implementation
#include <iostream> #include <vector> void heapify(std::vector<int>& arr, int n, int i) { int largest = i; // Initialize largest as root int left = 2 * i + 1; // Left child int right = 2 * i + 2; // Right child // If left child is larger than root if (left < n && arr[left] > arr[largest]) { largest = left; } // If right child is larger than largest so far if (right < n && arr[right] > arr[largest]) { largest = right; } // If largest is not root if (largest != i) { std::swap(arr[i], arr[largest]); // Recursively heapify the affected sub-tree heapify(arr, n, largest); } } void heapSort(std::vector<int>& arr) { int n = arr.size(); // Build max heap for (int i = n / 2 - 1; i >= 0; i--) { heapify(arr, n, i); } // Extract elements from heap one by one for (int i = n - 1; i > 0; i--) { // Move current root to end std::swap(arr[0], arr[i]); // Call heapify on the reduced heap heapify(arr, i, 0); } } int main() { std::vector<int> array = {4, 10, 3, 5, 1}; heapSort(array); // Print the sorted array for (int num : array) { std::cout << num << " "; } // Output: 1 3 4 5 10 return 0; }
Time and Space Complexity
Time Complexity
- Build Heap: O(n) - This is not the obvious O(n log n) that you might expect, but can be proven to be O(n)
- Heapify: O(log n) - Each call to heapify takes O(log n) time
- Extract from Heap: O(n log n) - We call heapify n times
Overall time complexity for all cases (best, average, worst):
- Best Case: O(n log n)
- Average Case: O(n log n)
- Worst Case: O(n log n)
This consistent performance across all scenarios is a significant advantage of heap sort.
Space Complexity
- Space Complexity: O(1) - Heap sort is an in-place sorting algorithm
- The recursive implementation of heapify can use O(log n) stack space, but it can be implemented iteratively to use O(1) space
Advantages and Disadvantages
Advantages
- Consistent performance: O(n log n) in all cases
- In-place sorting: Requires only O(1) extra space
- Efficient for large datasets: Guaranteed O(n log n) time complexity makes it suitable for large arrays
- Better worst-case: Has better worst-case performance than quick sort
- No dependence on input: Performance is independent of the initial order of elements
Disadvantages
- Not stable: Does not preserve the relative order of equal elements
- Not adaptive: Performance doesn't improve for partially sorted arrays
- Slower in practice: Often outperformed by quick sort in practice due to better cache locality
- Complex implementation: More complex than simple algorithms like insertion sort
- Poor locality of reference: Doesn't access memory in a predictable pattern, causing more cache misses
Comparison with Other Sorting Algorithms
Algorithm | Best Case | Average Case | Worst Case | Space | Stable |
---|---|---|---|---|---|
Heap Sort | O(n log n) | O(n log n) | O(n log n) | O(1) | No |
Quick Sort | O(n log n) | O(n log n) | O(n²) | O(log n) | No |
Merge Sort | O(n log n) | O(n log n) | O(n log n) | O(n) | Yes |
Insertion Sort | O(n) | O(n²) | O(n²) | O(1) | Yes |
Selection Sort | O(n²) | O(n²) | O(n²) | O(1) | No |
Compared to other sorting algorithms, heap sort has the following characteristics:
- Better worst-case performance than quick sort (O(n log n) vs O(n²))
- Uses less memory than merge sort (O(1) vs O(n))
- Unlike quick sort and merge sort, heap sort doesn't require recursion
- Often slower in practice than quick sort due to cache locality issues
- Doesn't preserve stability like merge sort and insertion sort do
Applications of Heap Sort
When to Use Heap Sort
- Limited memory environments: When memory usage needs to be minimized (O(1) extra space)
- Guaranteed performance: When a consistent O(n log n) performance is required regardless of input distribution
- Worst-case scenarios: When the worst-case performance is critical (better than quick sort)
- Priority queue operations: When building a heap is part of other operations, like in a priority queue implementation
- K-way merging: When you need to merge k sorted arrays (the heap can store the smallest elements from each array)
Real-World Applications
- K largest/smallest elements: Finding the k largest or smallest elements in an array
- Priority scheduling: CPU scheduling and other priority-based systems
- Graph algorithms: Dijkstra's shortest path and Prim's minimum spanning tree algorithms
- External sorting: Sorting large files that don't fit in memory
- Media streaming: Buffering and managing media data for streaming applications
Optimizations and Variations
Iterative Heapify
The recursive heapify function can be implemented iteratively to save stack space:
function heapifyIterative(arr, n, i) { let largest = i; let complete = false; while (!complete) { const left = 2 * largest + 1; const right = 2 * largest + 2; let newLargest = largest; if (left < n && arr[left] > arr[newLargest]) { newLargest = left; } if (right < n && arr[right] > arr[newLargest]) { newLargest = right; } if (newLargest !== largest) { [arr[largest], arr[newLargest]] = [arr[newLargest], arr[largest]]; largest = newLargest; } else { complete = true; } } }
Bottom-up Heap Construction
Starting from the bottom of the heap and working up (as shown in our implementation) is more efficient than inserting elements one by one. This approach has O(n) complexity rather than O(n log n).
Other optimizations include:
- Cache optimization: Modifying the heap structure to improve cache locality
- Parallel heap sort: Using multiple threads to process different parts of the heap simultaneously
- Weak heaps: A variant that reduces the number of comparisons needed
- Smoothsort: A variation that performs better on already sorted or nearly-sorted arrays
Conclusion
Heap sort is a powerful, in-place sorting algorithm with guaranteed O(n log n) time complexity. While it may not be as commonly used as quick sort in practice due to cache locality issues, its consistent performance and low memory requirements make it valuable in specific scenarios.
Key Points to Remember
- Heap sort has O(n log n) time complexity for all cases, making it reliable in worst-case scenarios
- It sorts in-place with O(1) extra space, but is not a stable sorting algorithm
- The algorithm involves two main phases: building a max heap and repeatedly extracting the maximum element
- Heap sort is less commonly used in practice than quick sort due to cache performance
- The binary heap data structure used in heap sort is also fundamental to priority queue implementations
Next Steps
To deepen your understanding of efficient sorting algorithms and related data structures, explore these topics:
Related Tutorials
Related Tutorials
Heaps & Priority Queues
Learn about the binary heap data structure that powers heap sort.
Learn moreQuick Sort
Compare with another efficient sorting algorithm with O(n log n) average time complexity.
Learn moreMerge Sort
Explore another divide-and-conquer sorting algorithm with O(n log n) complexity.
Learn more