The right way to build
large media websites

Laurent Mirguet

Media is a speed-driven industry in which technical roadmaps change every month or even week. Partly to accommodate worldwide tech leaders moves, and partly to tend to consumers entertainment demands, projects multiply with ever shorter deadlines.

As CTO and PM of the long-standing international media group aufeminin – created in 1999 it has now over 90 million users and 1 billion page views per month in Europe – Laurent Mirguet has direct experience in building large scale media websites in highly unstable environments. He first discovered specific media challenges at Yahoo. Working there in the early 2000’s, when Yahoo was amongst the largest media platforms worldwide, he soaked in the rewarding attitudes to adopt.

In this interview, Laurent shares his understanding of large website challenges, and the philosophy to follow for building massive scale internet websites in a fast paced environment.

Technology versatility is key

1) What flexible engineering looks like

“In the media industry, product and editorial strategies evolve at a very fast pace. Google launches Accelerated Mobile Pages, Facebook goes for instant articles, Android TV is coming up, Amazon has their own project and Oculus is developing… but don’t you forget about Pinterest either! At the same time, new content producers springs out of Snapchat or Facebook and become your competitors. On top of all of this, advertising clients are all too happy to work with you on never-done-before projects. People throughout the world are excited by novelty. If you wish to ride the wave, you need to be there as it happens. So there are always new distribution channels and content formats, and you need to take a stand on all of these concurrent projects. You need to display flexibility to manage so many different tech projects”, Laurent explains.

aufeminin brands

At aufeminin, some additional complexity comes from managing 20 websites in 8 different languages, with diverse integrations due to locally specific advertising legislations. So the problem is less about improving a technology at its very core, but rather permanently adapting to new distribution channels, market trends, and connections over a number of platforms.

2) A mobile force of a team

“Given our rapidly moving technological background, what I’m particularly interested in new hires is an entrepreneurial strand. When in an interview someone tells me ‘I have a small website that I’m trying to leverage’, I automatically think ‘now you’re speaking to me!’. Because this is how they learn how important paying attention to your audience is. What does it want? How can you make it grow? Developers with this mindset are the ones that will find what we do exciting”.

Laurent goes on “We’re definitely looking for engineers that show potential for being autonomous. A world expert in a specific PHP version would not be a good fit for our team, we deal with far too many technologies and changes.” Whether they’re working on PHP, ASP, Java or .NET, aufeminin engineers need to be flexible and to adapt. “We’re not building a technology, we’re building web platforms. It’s about the audience.”

Once you’ve got the right potentials, Laurent stresses the importance of instilling knowledge mobility in your team: “Training team members not to specialize is paramount. Of course experts do emerge on specific topics. One developer has worked a lot on search engines topics, while the other happens to have worked on advertising ones. But preventing specialisation to get too ingrained is crucial to prevent bottlenecks.”

One day your expert is going to leave and work for someone else, or he’s going to be on holidays, calling sick or working on another project that has priority. “It is bound to happen, and if no one in the team knows how to deal with his area of expertise, everything is blocked, sometimes for a while. So everybody needs to be able to work on any topic”, Laurent says.

“From time to time, you have to step in and prevent the search engine expert from working on an upcoming search engine task, but rather attribute it to the informal facebook expert. The latter one will come to the former one for advice when facing a roadblock. Otherwise one day you’ll have to put much code to the bin as understanding it would take much longer than rebuilding it from scratch.”

aufeminin banner

3) Modularity is key to infrastructure scale

Aufeminin tech team is building websites. And Laurent is pretty adamant that in this case, the issue is not about technology, but rather the way you structure it. “Upon joining aufeminin, I knew I would have to replace the ASP CMS existing at the time with a new one. The team was very keen on choosing PHP for the new framework, and it did not really matter to me, so that’s what we did. In my opinion, we could have developed the framework in java, .net, ruby or so-and-so, it would not have made any difference. Because in my experience, especially while working on internet platforms, programming languages and chosen technologies do not make much of a difference on your final product. When working at Yahoo, we made websites with C++ or Perl, and their audience was gigantic.” The only difference there is when choosing a specific technology over others is its recruiting traction as well as its access to a framework and library ecosystem that is more or less lively.

“Scalability is mainly an architecture topic for me. What needs to be done is getting the most basic building block working and then to be able to add multiple connections to it. If you’re currently working with 3 servers, you definitely should be able to plug in 4 or 5 or 6 ones. You should get the most standard possible setting up, so you can multiply it and be able to grow it very quickly when audience is rising.”

For most mass-market CMSes, there is the assumption that it will work with one web server and one database backing it up. This will typically become a bottleneck if this assumption is hard-coded in the system. “At some point traffic will have risen on your website, and you will need n machines to balance audience loads. You’ll need to assume that any request from the same user can be treated by any web server and that your data is split in many different databases. If your CMS was not designed for this kind of architecture, it can be very difficult to introduce this flexibility in the system without breaking everything.

“So you should build everything thinking you’re going to have lots and lots of users, lots and lots of pages, lots and lots of data to deal with. And how much lots and lots is does not really matter, just make sure that lots and lots of anything would be working. You will still encounter thresholds from time to time, especially some specific ones that you did not even think about, but it’s not that limiting. For example, you could have enough machines, but no more IP addresses to allocate.”

The second dimension of scalability for Laurent is about minimizing the impact of audience evolution with caching at many different levels. So for example, a website page requested by 10 users would indeed be calculated once and served 9 times. Cost is important for the first user and then cost becomes minimal. Laurent concludes that “scalability is not that complex of a problem for people understanding software architecture. On average, 6-9 months after joining aufeminin, a new developer knows how to deal with scaling for audience peaks.”

aufeminin banner

The one quality your product needs is client value

1) What you’re trying to accomplish

“Whatever the situation, do not work on any technical transformation that does not generate product improvements. This is the only way to justify the investment and time spent by your tech team”. Every single change should have an impact, otherwise it is of no use.

Laurent goes on to explain “What is the purpose of getting caught in an engineer frenzy for a new technology really? Hype is not a good enough reason for me, and there are no such projects in the company. You just spend large amounts of financial and human resources down the drain. Which is not to say that there should not be great engineer projects. For example, some of our developers are working on algorithms to better understand and leverage buzz phenomenons. Most of our content is seen an average of 200k times. But some of it is seen 20 million times, how come? It’s a real technological and conceptual challenge, and it’s also a real company challenge.”

This mindset shows in a variety of ways at aufeminin. Currently moving to cloud, Laurent is well aware of the technical advantages of it. Still, it won’t generate more revenues for the company per se, so Laurent does not want the project to generate additional expenditures. Also, when aufeminin tech team was working on building a new CMS with PHP, they chose to start working on the single page articles. Accounting for approximately 30% of aufeminin traffic, getting it right meant dealing with the most important company needs, ie advertising and service geolocalization. “Once this was done we knew we had the basics covered and we could move on one step at a time to cover other website sections.”

aufeminin banner

2) Ship it fast

Once the purpose of your development is clear, ask yourself what is the most efficient way to get there. This boils down to creating a quick-ship mentality in your team. “Would you develop the feature within your framework, which in aufeminin’s case is quite complex, or should you get out of it? It all depends on the project, a never-done-before campaign with one advertiser could be too complicated to integrate to your CMS, and it’s disposable developments. Then you’d better start from scratch building a drupal website that has the same look and feel than your regular CMS based website.”

Considering the pace of changes aufeminin is confronted with, they always want the shortest development cycle, and agile methodology is not enough. “We definitely have to question what features are really needed for users. For the company as a whole, we’re often better off releasing on a short deadline, with a thinner set of of features. Sometimes to get things rolling faster, you need to buy software or integrate with an external system. You’re still better off than reinventing the wheel, in which case you’re losing time and money.”

my little paris

It’s one big team

1) Drive projects that impact other departments

“Some projects come from other business departments”, explains Laurent, “and we challenge both the expected output and they way to get there. Some other projects are driven by the tech team, as it is the case for SEO for example.” So within aufeminin company, some room was given for innovation and projects thought and driven by developers.

Besides developer driven projects that impact other departments in the company, Laurent urges tech team to always remember to have an impact on other services work. “In each project that you do, make sure that the end result is going to be visible. And by visible I mean intelligible and impactful for them. Going back to the example of rolling out a new CMS, the project reduced infrastructure costs and divided time to generate a page by 2.5, but we made a point to work on page redesign so that the change would be visible to anyone. Rolling out this project would take a full year, we had to make sure other services could feel and understand what we were working on during all this time”.

2) Make sure your team gets others’ concerns

“I’ve already mentioned the importance of looking for developers that have an autonomous streak. Once they join your team, you can further foster autonomy. Start with the ones that show the most potential, the ones that are already gradually able to take ownership of projects. Of course you should not send them right in the middle of the battlefield otherwise they’ll get burned while interacting with other services, never wanting to get back there again.” So you should work hand in hand with them, progressively giving them more freedom.

At the end of the road you want every single developer in your team to be able to bring a new feature to life. This starts by being able to discuss intelligently with the clients asking for a new feature. “If our journalists ask for a new type of reporting in their admin, I want all developers to be able to go and discuss with them what it is they really need, be able to understand it and to define the feature specs.”

Love this? Share it!

Related Posts

Get notified when new content is published