Added 1.5.16 (Updated 6.10.17)
Is al-Qur'an 4:101 really calling non-Muslims "open enemies"? And doesn’t 8:12 say to kill the infidels?
Both revelations are specific and do not apply in general. Both are in reference to circumstances at the time of the Prophet Muhammad ﷺ. Let me explain the background, and then I’ll get to the specifics. Most importantly, I'll add include the statements of Allah and the Messenger ﷺ that do apply generally and presently regarding how Allah demands Muslims treat non-Muslims.
The people of Makkah, the Quraysh, had turned the Ka’bah into a place of idol worship, and since people made pilgrimage there, this became part of their economy. When the Prophet Muhammad ﷺ came with a revelation saying to worship none but Allah, and especially when people started following him, the Quraysh objected. They did not want any competition for their way of life and the religion of their forefathers. More than that, they persecuted those early Muslims. They tortured, killed, lied about, and boycotted them. The boycott, in particular, went on for three years wherein the Muslims were cast out of their homes and were trapped in a valley. At a point, they resorted to eating leaves and camel skin. Eventually, the Muslims migrated north to al-Madinah (Yathrib) to escape that persecution.
Ayah (sign, literally, and, here, akin to verse) 8:12, the second ayah mentioned in the question, was revealed when 1000 Quryashi fighters had come to al-Madinah and were descending on virtually the only 300 or so able-bodied Muslim men on the face of the earth. The ayah tells those Muslims in al-Madinah, in that situation, to fight and kill members of those specific Qurayshi warriors. It does not tell Muslims in general to kill non-Muslims in general.
As for the first ayah, 4:101, the reference to "open enemies" relates specifically to disbelievers who fit the condition laid out in the second part of the same ayah, which says, "if you fear that those who disbelieve may disrupt (or attack) you."
More specifically, it is in relation to the speakers in the following hadith—those, who say they should have attacked the Muslims when they were praying: Narrated Abu Ayyash az-Zuraqi: “We accompanied the Apostle of Allah at Usfan, and Khalid ibn al-Walid was the chief of disbelievers. We offered the noon prayer. Thereupon, the disbelievers said: 'We suffered from negligence; we became careless. We should have attacked them while they were praying.' Thereupon the verse was revealed, relating to the shortening of the prayer (in time of danger) between the noon and afternoon prayers" (in reference to al-Qur'an 4:101, Abu Dawood 4:1232).
So the original open enemies are persecutors who are trying to sneak up and kill the early Muslims while they were praying. Those those who hate and have the desire to harm or kill are enemies. It's pretty obvious, actually, even if they're from your own family, which, unfortunately, was the case at that time, in the tribal sense. That's maybe why it's necessary to say. So a more general wisdom is awareness that, as humans, we're not always willing to admit who are enemy is. And that is a basis for the game of ash-Shaytan (Satan), who Allah also calls a sworn and open enemy (see al-Qur'an .
However, Allah did pronounce the ideal, default position Muslims should take toward non-Muslims. The first of the following ayatayn (two ayah), the second one describing the position toward those who mean dire harm: "Allah does not forbid you from those who do not fight you because of religion and do not expel you from your homes—from being righteous toward them and acting justly toward them. Indeed, Allah loves those who act justly. Allah only forbids you from those who fight you because of religion and expel you from your homes and aid in your expulsion—(forbids) that you make allies of them. And whoever makes allies of them, then it is those who are the wrongdoers" (al-Qur'an 60:8-9).
Further, even when the Muslims were in governance over a diverse population, the Prophet Muhammad ﷺ said, “Whoever kills a mu‘aahid (a non-Muslim living under Muslim rule) will not smell the fragrance of Paradise, although its fragrance may be detected from a distance of forty years” (Bukhari 2995). In another narration he said that whoever harms such a non-Muslim “harms me and whoever harms me annoys Allah.” In yet another he said, “He who hurts a non-Muslim living under Muslim rule, I am his adversary, and I shall be his adversary on the Day of a Judgment.” The Prophet ﷺ put himself between the other Muslims and the non-Muslims. Subhan Allah (glory to Allah).
Someone will rightly ask, "But what happens when it comes to violence?" Because it did pretty frequently come to violence during the lifetime of Prophet Muhammad ﷺ, who was a military leader among his other roles. There's no hiding that. His life provides examples for all aspects of life. The rules revealed through the Prophet Muhammad ﷺ are much more restrictive than the Geneva Conventions. Killing unarmed non-Muslim targets graphically departs from the actions of Ahlus-Sunnah wal-Jam’ah and whoever does this stands in violation of Shari'ah, the Law of Allah.
In some circles and literature (mostly aged), the military legacy of Prophet Muhmmad ﷺ is disparaged as unholy, while in other circles, still in 2022, they say Vive l'Empereur!, literally praising Napoleon Bonaparte (last name to disambiguate from Napoleon Dynamite, who has some positive qualities). It's a bit glib, perhaps, but even such outrageous comparisons may provide serious insight.
The treaty of Hudaybiyyah would make peace between the Quraysh, who were mainly in Makkah at the time, and the Muslims, who were mainly in al-Madina, at the time. A central feature was laying out some means of sharing access to al-Ka'bah in Makkah. During the drafting, meaning before anything was signed, the Quraysh demanded the words Bismillaahir-Rahmanir-Raheem (in the name of Allah, the Most-Gracious, Most-Merciful) be removed from the top of the document. The Prophet ﷺ agreed that the words be removed despite protests from prominent companions of his. The Prophet ﷺ also signed treaties with the Jews of al-Madinah. Unrest was perhaps even partly generated out of a paranoid pre-emptive sort of defensiveness among the Quraysh and those particular Jews, though members of both also reverted to Islam.
One prominent case where the Prophet ﷺ really just reached out was prompted by a Jew who had recognized and accepted the identity of the Prophet ﷺ based on true prophecies that had remained in the partially corrupted Torwraa' (Torah). The new convert was prominent among those Jews, to the extent he was a highly respected scholar. He told the Prophet ﷺ to gather the other prominent Jews and to ask them about him.
Many deeds widely reported regarding Muslims are of the deeds hated by Allah, and they tragically complicate life for all, including Muslims. And in places, the foremost damage falls upon Muslims. Ninety percent of ISIS's victims are Muslims! When military action is required to defend against oppression, whether that oppression be on Muslims or non-Muslims in Muslim protection, the Shari’ah demands firm restrictions, restrictions ISIS wantonly disregards, just to take them as an example of the ugly machines we've seen in the media with the name Islam slapped on them. Saudi Arabia's monarchy is absurdly corrupt and misguided, though there's a niche some scholars carved out originally for al-Hujjaj. This is because even while he did some bad things, he also did some good things.
I was in Saudi Arabia. Only two weeks, but that's plenty to be impacted in a place like that. Despite lack of proper recycling, despite racism in the workplace, despite lasciviousness in the palaces, despite passport control and abuse of the lower bracket of expatriates, despite gender quandaries . . .. Despite all that and more, there are some things I appreciated.
The difference became most stark to me on my way back home. The vibe in the JFK airport terminal was so heavily commercial that it overwhelmed me. I was sort of shaken by it due to the contrast against my environment the prior two week.
In Saudi, I walked around not feeling I was in a marketplace. Until, of course, I did walk into a marketplace, and even then it didn't feel as imposing a US marketplace. And once I went back out of the marketplace, I was out, with mental sky in which to paint my thoughts without offers from companies who want to sell me something blasting through the clouds to steal my thunder. There's a joy and peace when a person feels unmolested by advertising. That's wonderful. And that's just an example of beauty in that culture, at least in my brief experience. My point isn't to analyze Saudi Arabia. It's just that in raising the anti-Islamic nature of its monarchy, I didn't want to make it sound like there's nothing good in Saudi, because there sure is. The people in al-Madinah, specifically, are amazing and with much love and light.
So there are more and less accurate examples out there in the news which have served as demonstrations of what Islam is, especially in circumstances of combat. As for what Allah and the Messenger ﷺ say, foremost, killing of non-combatants is forbidden. That comes with much emphasis on safeguarding women, children, and the infirm. Even when the Prophet ﷺ had fallen ill and died, the first khalifah (ruler of the unified Muslims), Abu Bakr, Allah be pleased with him, emphasized specific restraint upon the military leader Usama. The great companion said, “Do not kill an old man, a woman, or a child. Do not injure date palms and do not cut down fruit trees. Do not slaughter any sheep or cows or camels except for food. You will encounter persons who spend their lives in monasteries (i.e., peaceful Christians). Leave them alone and do not molest them” (qtd. in The First Caliph, and preceded by similar statements of the Prophet, himself, ﷺ).
Even within these limitations, personal anger is an invalid intention for lethal force as evidenced in the famous narration in which Ali, the fourth khalifah, Allah be pleased with him, was about to prevail in a fight when he tossed his sword aside in forfeit. Ali's opponent asked why he hadn't finished him off. Ali indicated he'd quit when when the man had spat in Ali's face. It had changed Ali's niyyah (intention) due to his anger at being spat upon. In Ali's heart, the sword fight shifted away from being about necessary defiance of tyranny and defense against extermination. His anger stemmed from his own, personal, motives. He couldn't kill a man for those reasons.
Anyone who behaves rashly and kills citizenry has violated the Shari’ah and is going against the way of Prophet Muhammad ﷺ.