Some people are trying to pull team down: Ravi Shastri