Thomas Raef from We Watch Your Website argues that every hosting company should be analyzing access logs and scanning for threats in real time using AI, not waiting until a site is already compromised to review what went wrong. In part two of his conversation with Jesse Friedman, the two explore what it would actually take for hosts to bring that capability in-house and why farming security out to third-party vendors is not always the most cost-effective path.
The conversation turns to a thorny question at the heart of WordPress security and open source. Raef has built AI-powered skills files that detect vulnerabilities, but open-sourcing them could give hackers a roadmap. Jesse suggests that organizations like the Secure Hosting Alliance might provide a trusted channel for sharing these tools among verified hosting companies without exposing them to bad actors.
Beyond detection, Raef reveals that his plugin analyzer can produce corrective code when it finds a vulnerability, potentially cutting the WordPress plugin review cycle from weeks to days. Jesse and Tom discuss why this kind of contribution could be a game-changer for the volunteer-driven plugin review process, and why fixing the source of vulnerabilities is always better than handing out band-aids after the fact.
Links:
- We Watch Your Website
- Secure Hosting Alliance
- Patchstack
- Jetpack
- Wordfence
- What the Frick Is Managed Hosting (Season 1 Episode)
Transcript
Jesse Friedman: Welcome to Impressive Hosting, a podcast about the role hosting plays in shaping the open web. I’m your host, Jesse Friedman. On the show we go deeper than uptime and dashboards. We talk about hosting as infrastructure, about ownership, independence, and what it takes to build ethical, high-end WordPress hosting that actually serves creators, businesses, and the internet itself. Before we dive in, head to impressive.host, that’s where you can comment on episodes, ask follow-up questions, and help shape future conversations. You can also find links to follow, like, and subscribe wherever you listen. Today our guest is Tom Raef from We Watch Your Website. This is part two of our conversation around protecting the open web and creating better security standards. Where we left off was in talking about how AI can be helpful in patching and also identifying vulnerabilities inside of plugins. One of the things we identified there was that a lot of times people will use AI and other things and they’ll find, or, you know, other tools and they’ll find possible vulnerabilities. Things that look suspicious, but actually aren’t, and that you mentioned Tom, is kind of like a detractor. It’s taking away from your ability to actually work on true vulnerabilities, and that can be a lot of noise. You had mentioned that you’re building an AI tool to help in identifying those vulnerabilities on plugins. And then we talked a little bit about the fact that there’s 60,000 plugins out there and, and then I mentioned that there’s actually 60,000 plugins times however many versions of plugins there are out in there in the wild. And you know, we’re kind of going back and forth on this idea around like AI’s ability to help hackers. Can we use that same information to help strengthen and harden security? But you know, I think one of the things that, you know, before we dive back into AI, one of the things I want to kind of push on here is this around why is it that we feel like customers don’t want to update? We talked about it very briefly last time, and I think from my perspective, what I’ve seen over the years is that customers can get very set on a way of working, which is basically set it and forget it. They build a website, it’s working perfectly. They don’t understand how things can change because browsers evolve or markup changes or standards, whatever. And they get this feeling that if they change it, if they touch it, it’s going to break. And I think that’s kind of complicated because, you know, you don’t want to, again, we don’t want to instill fear, we don’t want to make customers feel like things are overly complicated, but at the same time, we need them to be doing some things there and taking action. So, you know, when we think about a host’s responsibility to keeping things up to date, you know, I firmly believe that the host can take a lot more of a curated approach. Take a little bit more of a hands-on approach. What we do here at WP Cloud is that any one of our partners, who’s using the WP Cloud infrastructure, knows that their entire platform is up to date. We keep the stack up to date. We only support the latest versions of PHP. We keep WordPress up to date, and it’s forced, updated rather quickly. We keep plugins up to date. And why do we do this? Well, we do it because it’s the hard part. We want to make easy for end customers and the fastest way, and the best way to make it easy for end customers is frankly just to do it for them. And now I’ll tell you, there are customers out there who get super nervous about this. But what we found is that for the vast majority of them, if you take the control out of their hands on something like this and you just tell them, this is the way it works, this is the way it operates, we’re going to update this for you. It actually builds confidence in them, in us, and taking care of that action for them because they don’t actually feel like it’s kind of like a willy-nilly thing that they’re not taking seriously, that they’re not going to update it and break my site that they’re actually going to do something about it. And that’s what we do with WP Cloud. We, you know, if we keep things up to date, we’re also doing a good job of making sure that we’re not breaking anything. And if, and if there is a break, we roll it back. But I’m curious from your perspective, I’m like, how much do you think it would help the overall market, the overall WordPress ecosystem, if hosts kind of enacted the same policies that WP Cloud does in that it keeps everything up to date, whether the customer wants it or not.
Tom Raef: To me it, that has to be the industry standard. You know, if you’re going to say that you’re offering managed WP services, everything that you guys do has to be included. And I’m definitely not your typical, you know, website owner. But when I think of managed WordPress, that’s what I’m expecting, you know, updates, you know, you keep things up to, so I don’t have to worry about anything other than maybe writing some blog posts, you know, uploading some new pictures. Just the stuff that a website owner should worry about. So, yeah, I believe it has to be the industry standard. And I know there was discussion about even using the term, you know, managed WordPress. And I believe that, you know, I think a lot of that, some of that angst that came out over the term managed WordPress may have been just because so many people offer it, but they’re not doing it. You know, you manage WordPress. Really? What do you do? You, you know, you make it so that I can spin up a new version of WordPress for a new domain, you know, quickly. That’s not managed. That’s just, you know, you’re offering free installation service or something. I don’t know it, but yeah, I firmly believe
Jesse Friedman: Well, we actually, we had an episode last year on season one called What the Frick Is Managed Hosting. And that was because Jess Frick from Pressable at the time she had come on the show. But we wanted to address exactly that, which is that the term managed hosting doesn’t necessarily mean the same thing to everybody. And so we actually dedicated a decent part of season one to identifying what exactly is managed hosting. And it was a question that we asked a lot because I think hosting companies have this mentality of, you know, it’s a buzzword, it’s a jargon, so we will throw it on our website, but if you’re not clearly defining it, then it actually kind of dilutes the meaning of it across the board. So, you know, it’s, you know, I agree with you completely that if you’re offering managed hosting, whether you’re offering managed hosting or not, frankly, I think it’s something that should be done because I think at the end of the day too, if you are, if you’re offering shared hosting and you’re putting WordPress on a shared box, I could be wrong about this, but I think the majority of those customers are actually, you know, the ones paying two, $3 a month for unlimited sites on a shared box are probably the ones that need the most help managing things anyway. So, you know, some aspect of this I think needs to trickle down to basically all WordPress hosting.
Tom Raef: Right. Yeah. It, and you know, like I said, there was some angst last year, whatever, about just using the term WordPress and, yeah. To me, you know, if you’re going to include that in the name of your service, then you have to support it. You know, you have to manage it. And managing it means, you know, keeping plugins up to date and things like that. Themes even, you know, the whole, you know, you have to monitor things, you know, in our world. The key terms that people like to used to always throw around was heuristics. We do heuristic scanning and that. I used to love going to different shows and stuff, and I’d ask people, so what is, what do you consider heuristic scanning to be? Well, you know, it’s like a deeper level and you know, you’re like, okay, so you have no idea.
Jesse Friedman: Right, right.
Tom Raef: So, yeah, you know, we’ve got our terms in the security world. You guys, you know, and on your end have your terms. But yeah, I think that managed WordPress has to be, you know, all encompassing, you know. So I.
Jesse Friedman: Yeah.
Tom Raef: So I.
Jesse Friedman: If we think about that from that perspective, what position should a host be playing into with AI and managed hosting? Should AI be something that a hosting company is using regularly to, you know, look for infiltration. Should they be relying on a security company like yours, Patchstack, Jetpack, like, you know, because of the fact that the majority of our audiences are people working in the hosting industry, I’m asking because what do you think they should be hearing from you in terms of how they should be using AI to secure websites?
Tom Raef: I honestly believe, and, you know, I’m a vendor, but I honestly believe they should be doing it in-house. You know, and I tried that years ago, reaching out to some of the hosting providers like, Hey, look, you guys pay me as a consultant. I’ll come show you how to set this up internally. And you guys run it in your data centers and you know, write the program interfaces and so on. So, but I’ll show you how to get all the information you need. And they’re just like, nah, that sounds too expensive. Like, okay. But, so I think it boiled down to, I probably didn’t do good enough jobs selling them on why they needed it, but they really do. I mean, you have to know what’s going on. You know, if you’re not streaming and analyzing access logs in real time, you have no clue. You know, were you going to look at them after a website’s been infected? Oh yeah. Look at this. Oh wow. We probably should have caught that, huh? I mean, you know, to me they should be doing it. And if an agency is offering, you know, full managed, WordPress services. To me it has to be part of that as well. I mean, farming it out, you know, to a vendor. That’s okay I guess, but then it’s truly a full expense. You know? I mean, you’re paying, you know, possibly per website, per server, per whatever. But, you know, it’s something that you could do internally. And with AI for, you know, some people say, well, then I got to hire staff. You know, if I do all that internally, then I got to hire staff. Not necessarily. I mean, with, like I said, with AI, you definitely don’t have to hire, you know, a whole team of people. You could have AI implemented at the core and let it do the crunching, the sending of, you know, alerts, whatever. I mean, the system is you, you can have it right there. So.
Jesse Friedman: One of the things that I worked on years ago, I mentioned BrewProtect in the last episode. This was a company I co-founded. We ended up bringing to Automattic, and the way it worked was that we identified attack vectors from across the internet trying to infiltrate. Brute force into websites. But the magic was that we would update our blacklist, these attack vectors in real time and then distribute them via the plugin. And so it was hosting agnostic. The hosting company would update their own blacklists, but usually overnight. So they didn’t have this real time data. But the benefit was, is that we could learn from attack vectors that were hitting, you know, a site on host A and then use that to protect a site on host B. That was actually, you know, a really wonderful way to use cloud-powered data and share it among even competitors. And it makes me think, because if you’re suggesting that every single hosting company should be doing this in-house, that is an investment. That’s an investment in time. It’s an investment in the updating of, you know, the skills, all these things. So it makes me think, is it, would it make sense then for someone like yourself to be building a skill? An AI agent skill that could be leased, borrowed, you know, open-sourced to hosting companies. So that way, you know, when it comes to checking logs, when it comes to looking for vulnerabilities, they don’t have to train their agent every single time they can. They can rely on maybe some things a little bit more like a open protocol. But that leads me to another question. ‘Cause that, to me sounds like something easily commercialized. Like you could sell that, you could go to hosting companies and tell them how powerful this is. But you know, we work in WordPress, so our default is to open-source things. What do you think the ethical answer is there? If you were to open-source your AI training skill on how to detect these things. Would that just make it easier for people using AI to hack to know what to look for and maybe get around these things? How do you balance that when you’re trying to, you know, stop people from using the information you’re using ethically to, you know, unethically?
Tom Raef: Right. Yeah. And that’s something, you know, I come across, runs through my brain all the time. You know, I could just open source this, you know, if, if I’m looking at, if you know me being an altruistic, you know, individual, I could just open source this and millions, and millions and millions of websites will be safer because of it. But open source means it’s open source, which means hackers get it. So, you know, when you’re talking about a skills file, we have, we have the skills file that we use
Jesse Friedman: I imagine you do. Yeah, sure. Right.
Tom Raef: We have numerous skills files, but you know, to open source that Yeah, now the hackers know what you’re looking for. Not that it’s any big secret, but I mean, it kind of is like my brain’s just wired. Like I have no, I love music, I love art. I have no talent in those areas. None. I did a website years ago and my wife, who’s very artistic, looked at it and she goes. You are not going to put that on the internet, are you? But so I have this innate ability where I can see things and go back to the lawnmower days, and analyze and put together how it happened. So, like I said, we have these skills files. I don’t want hackers to have access to them. So if somebody can show me a way where I can open source my stuff, but not give it to hackers, I’m all ears. I really am.
Jesse Friedman: Yeah, it’d be interesting. Not an open source licensing expert, but I wonder if there’s some way to make something available, but put some kind of ethical line in the sand. I don’t know where you have to verify who you are and you promise to protect the information or something like that. I don’t know. I mean, it doesn’t matter though because all it takes is one person to publish it. Right. And then all of a sudden it’s available to everybody. Yeah. It’s an interesting problem to have.
Tom Raef: Claude was just, you know, released into the wild, right. Recently, you know, they released the source code for Claude. So, I mean, you know, how can, what do you do? You know, I don’t know. But I also
Jesse Friedman: Yeah. But it is interesting because I think the easier you make these things, then you think about, well, are you actually aiding the hacker? Well, you may not be aiding them in the sense that like you’re giving them a way in that they otherwise wouldn’t have figured out. But you are probably making their testing even faster, right? Because then they can try and do a bunch of things and then just ask the skill if it detected it, you know? And even probably have it running in real time and just have warnings go off when it’s detected and then you find another way around, it’ll just accelerate them in their efforts.
Tom Raef: Right. Yeah. People have asked me for years. You know, why don’t you have a thing on your site where you can, I can upload a file and it’ll tell me if it’s malicious or not. Like, because that’s going to be a total testing ground for hackers. I mean, they find your site, they upload a file. Oh yeah. It detects it, or Oh, no, it doesn’t. Look at it. I made these changes. No, it doesn’t detect it. Bam, let’s send that one out to all the websites we have control of. So, yeah, it’s, you know, I don’t know. Like I said if somebody could sit down with me and explain a way that I could open source my stuff, I’m all ears. I really am.
Jesse Friedman: One thing that comes to mind for me, which is interesting is, you know, Automattic is a member and a sponsor of the Secure Hosting Alliance. If anybody at home is interested, you can go to hostingsecurity.net. It’s basically an amalgamation of hosting companies coming together to stand up and say that we are practicing ethical standards and securing our infrastructure. They also do a lot of work around governance and making sure that, you know, our hosting company or the open web’s best interests are served in law making and things like that. I wonder if a group like that might be a great place to start where, you know that these are hosting companies who are invested in the securing of their customers and their websites. They do have ethical guidelines that they abide by and that they’re verified by this group. Maybe there’s something there. Maybe joining that group and then making that available to those hosting companies in that way would be helpful. And I think this is one of those situations where, you know, this is a complete side note, but I think it’s worth talking about, is that everybody always asks these questions. If I’m open-sourcing something, how am I going to make money with it? And Tom, I bet you, I bet you a million bucks. Just grabbing a random number, but I bet you if you were the expert who was providing open-source skill sets for AI to protect sites, that you’re also going to be seen as the expert to call when something’s harder to solve, right? Or whatever. So yes, you might be giving something away, but at the same time, I think it validates your expertise and, brings customers to your front door. And I think that’s kind of the power of open source, right? Like, yes, you’re giving something away, but you’re also going to be bringing in a lot more customers at the same time too.
Tom Raef: I think there’s a company you’re probably familiar with. They’re open source and I believe they have a market cap of, I don’t know, four or $5 billion. They have a .com and a .org. Heck see. I can’t think of the name of it. But anyway.
Jesse Friedman: Really. I’m sure you can’t.
Tom Raef: One of the things I was thinking of from our previous episode is, you know, I think a lot of time, you know, you mentioned about how people don’t want to update plugins because they’re afraid it’s going to break something. But I think the also the other thing I was thinking of is they also don’t think that their website’s going to get hit. I got a small mom and pop shop. You know, what do they want with me? So right. I don’t need to worry about all this. It’s the same reason why they don’t pay for security. You know, I don’t need it, you know? Nah, they’re not going to find me. So.
Jesse Friedman: That brings me back to another problem, which is the race to the bottom on pricing. Because of the fact that hosting companies have these super slim margins and they’re trying to constantly, you know, cut out all of these features in order to get down to the $2 a month hosting. They end up putting security into an add-on, you know, a bolt-on. Which is why I love, you know, I mentioned it before, but, and I am biased, but like Jetpack offers a lot of free security services, which I encourage hosting companies just to just make it available to those customers at, you know, out of the gate. You know, we no longer offering vanilla WordPress. Nobody is, you know, it’s very hard to find vanilla WordPress on a hosting scheme. So, you know, most customers are used to a plugin being included, right? So. Bring in as many security features as you can and pre-configure them. I think that’s the step that people miss is that, you know, you look, whether it’s Jetpack or Wordfence, or whatever it is that you’re looking at, it usually requires some level of configuration. And if you’re not doing that for the customer, but you’re installing it for them, you know, you’re missing a big thing here is that you’re installing it for them because they don’t have the expertise and the knowledge that it’s important enough to do themselves, but then you’re not configuring it for them, so then it’s not on and it doesn’t
Tom Raef: Right.
Jesse Friedman: So, yeah, it’s definitely something where a hosting company I feel like has a responsibility to step in more. And I think that selling, you know, security features that are table stakes for being on the web actually does have, you know, a detriment to the WordPress ecosystem as a whole. We talk about this all the time, right? Your WordPress ex, your first WordPress experience can be your last WordPress experience if it’s not a good one. And I think, you know, getting hacked is enough reason for people to feel like I might want to try something else. So, not to mention the overhead and the cost for a hosting company to have to manage that. Yes, they might outsource the remediation and tell the customer they have to pay for it, but let’s not discount the fact that every support ticket has a multi-dollar, you know, cost to it. If you’re charging $2 a month and you’re getting a support ticket once a month, you’re already. You’re already losing out, right? So especially with security tickets, they’re going to take even more time, a lot more overhead. So in the end, I think it’s actually, you know, quite valuable to customers. And if we can get ourselves out of this race to the bottom on pricing, then we can provide fully managed WordPress at a cheaper rate. We can provide these security features at a cheaper rate and help people to have a, you know, a much better experience.
Tom Raef: Right. Yeah.
Jesse Friedman: Yeah. So.
Tom Raef: I agree. Yeah. The race to the bottom is no, nobody’s going to win.
Jesse Friedman: Right. Yeah. So tell me, what is your thought process on giving back to the WordPress community? One of the conversations that we had early on was around utilizing the AI tools to help. You know, we talked last episode about the actual review of a plugin. So someone writes a new plugin, they go to put it on the WordPress directory. Maybe they’re not even following the WordPress guidelines, they get a no that you’re not going to be accepted. You mentioned that you actually put in those guidelines into an early warning system or a skillset or something where you can actually tell them, Hey, you’re probably not going to qualify to be listed on the WordPress directory ’cause you’re not following these guidelines. But what about all the other plugins that are already out there that have been up there for a long time. Do you think there’s a way to solve for that?
Tom Raef: Yeah, I mean, it, when I first started this, I said, you know, we do more than just WordPress websites, but obviously WordPress is, you know, the largest platform out there. With our plugin analyzer, my whole intention was to make it open source or make it free, because as I understand it, and you know more about the ecosystem than I do, the people who are reviewing plugins now, they do that on a volunteer basis, right.
Jesse Friedman: I mean, a lot of what goes into making WordPress possible is people volunteering. There are some companies out there who sponsor folks, but for the most part, yeah, it’s very community driven, volunteer driven.
Tom Raef: So I was like, you know this, I know you know it. It had been asked last year, you know. People need to, or have been stated, you know, people need to contribute back to WordPress. And I kept thinking, you know, God, what the heck can I do? Can I make my service free? That doesn’t, like you said you, that doesn’t pay the bills. But if I could come up with something like this where, and I’m not trying to eliminate, you know, the volunteers, maybe some of those people really enjoy doing that. I don’t know. But if I could offer something.
Jesse Friedman: You won’t be eliminating them. I think you’ll just be giving them the ability to breathe a little bit.
Tom Raef: That could be. But you know, I mean, when you see something where it take, where people are complaining in forums about, you know, I submitted my plugin, you know, a month ago and I still haven’t heard anything back. You know, I mean, to me that’s just crying for a solution. And you know it, I would like to present this to the powers that be and see if that is something that they would like to try to implement. You know, and I’ll just say, here you go, I just contributed, you know, not saying it’s going to be my only contribution, but you know, to me it could be a big one, you know, if you could shorten that cycle from, you know, 30 days of people not hearing stuff to, you know, I don’t know, like two weeks. We’ve already run some pretty big plugins through our system, and I think the longest it took was like, it’s like 900 seconds to fully analyze. So, and like I said, the part to the
Jesse Friedman: 900 seconds for the, for a single plugin or.
Tom Raef: Yes. But you know, I mean, this could be put on, you know, a bunch of different servers and, you know. They all pull from a queue and just keep crunching, crunching away. But the thing that we’ve built into this, that I think is really key is it produces code that will fix, if it finds a problem, it produces code that’ll fix the problem. So.
Jesse Friedman: And I think that’s a huge, huge win because if we think about the way in which the WordPress community. Or rather, the plugins that exist out there are written by the WordPress community. A lot of times it’s not by companies who have large staffs. A lot of the plugins are written by individual contributors, people who are just trying to, you know, take something they built for their website or for their client and make it available to the rest of the world. And they don’t necessarily have a group of people to constantly be updating and helping with these things. So I love that because it’s not just. The conversation I was having internally at work today too, is that, you know, a spellchecker, for example, will tell you that you got something, you spelled something wrong. And if you take the time, you can actually read the change. But for the most part, like autocorrect or whatever will change the way I spell. I don’t know, some, there’s some words that just are still beyond me, my dyslexia. But like autocorrect just fixes it, right? To me, that’s akin to cars that have the self-driving or self-parking features, right? Like it’ll parallel park for you. And I think if we see that being ubiquitous, we’ll actually see the skill of being able to parallel park go away. Right? I think what’s really interesting about AI is, is that instead of it, you know, you can train it, you can do this, you can ask your AI that say when it’s spell-checking, when it’s grammar-checking, when it’s checking a code on a plugin. Correct. Not only correct it for me, but teach me why. Right. So, you know, I think the best versions of skills that I’ve seen around helping people be better writers through AI. Is not just that it makes suggestions, but it also helps you understand why, you know, what is the English rule that you didn’t follow or why is this best practice or, you know, whatever it might be. But the same thing goes with code. If you can turn back to them and say, this isn’t just a threat. Now here’s an issue or something like that, but here’s why you want to change it and here’s the best, you know, things to do, like, you know.
Tom Raef: Escaping characters
Jesse Friedman: Escaping your characters or something like that. Right, exactly. That’s what exactly what’s coming up for. I had the function, the double underscore in my head, but I couldn’t remember what it was. But those types of things can actually come back and help you become a better programmer and a better developer. And so I think if we’re smart about the way in which we utilize these tools, not only are we going to improve and harden the security of all these things, but we can actually move a lot faster. Learn from these things so that the next time we write a plugin, we’re actually implementing those things right away.
Tom Raef: Right.
Jesse Friedman: That’s,
Tom Raef: Yeah. So like I said, you know, that was my goal was to make it full cycle, you know, where it will produce, you know, the correct code for you, but then also make this a contribution to the WordPress ecosystem and you know.
Jesse Friedman: I think that’s really great and I think one area too where you could be really helpful in assisting the community. You mentioned in the last episode the importance of logs and data. I think it would be really interesting that if we got to a place where this was implemented and helping to support the securing of these plugins. It’d be great to have an end of the year report. Understand, you know, how we, how we improve things. What were the most common vulnerabilities, what things can we put in practice to help ensure that, you know, people are writing better plugins, but also what can core, what can hosts do to surface these things early? By providing a better security user experience rather than just relying on third-party tools to solve it all.
Tom Raef: Right. Yeah, I agree. You know, like I said, I have nothing against the third-party tools at all. But you know, same time, you know, when you think about it from a logistics standpoint, you know, why not fix the problem rather than putting, giving everybody a band-aid, you know, why not fix the problem and be done with it? Or some, one of my. People said something about, oh, it’s like curing malaria. You could either give everybody unlimited bottles of DEET or you could come up with a way to kill mosquitoes. That’s
Jesse Friedman: Yeah.
Tom Raef: Yeah, okay. Yeah. Let there, there, there’s logic in there somewhere, but yeah.
Jesse Friedman: Yeah, no, I know what you’re saying. That you can either address the problem by trying to fix the symptom or you can address the problem by fixing the source. All right. Well, listen, this was another great episode. It was a lot of fun talking to you. I can’t wait to see what you are able to do to help out the community and empower others. I would love to see, you know, how you, maybe you approach helping hosts too. Maybe that’s something I can help you with is putting you in contact with hosting companies to see how we can get them leveled up in this world of AI. Tom, thank you so much for joining. It was a real pleasure.
Tom Raef: Thank you very much for having me. And, yeah, we’ll, I’m sure we’ll be bumping into each other somewhere around the world.
Jesse Friedman: For sure. Definitely.
Tom Raef: Thank you very
Jesse Friedman: Take care.





Leave a Reply