Click here to Skip to main content
13,861,988 members
Click here to Skip to main content
Add your own
alternative version

Stats

4.2K views
52 downloads
3 bookmarked
Posted 13 Feb 2018
Licenced GPL3

WpGet: Private wordpress plugin repository

, 13 Feb 2018
Rate this:
Please Sign up or sign in to vote.
Implement a private Wordpress reposiotory using Angular and PHP slim framework as backend.

1. Introduction

This article tells about I design and build wpget, the on-premise wordpress plugin repository. This is an interesting implementation that can be used as reference for api-oriented web applications. It is based on php stack, using slim framework as backend, token based authentication with the angular2 frontend. I will discuss about  backend and frontend design, having a deeper investigation about continuous integration, and delivery to the on-premise instances.

 

2. Why we need a private repository for wordpress

Wordpress is the most used CMS in the world with the most important plugin system. It can be customized with more than 5000 plugins and cover the 60% of CMS powered websites (29% of the whole websites). It's easy for a developer to create and publish a new plugin into the public repository. But what if i had to create and maintain a private plugin? Im speaking about:

  • a plugin with an high intellectual content,
  • a plugin not ready for the public, maybe focused on some particular business case
  • a custom super-plugin built used to inject custom plugin dependency

There are actually these opportunities into public repository:

  1. Make a free plugin
  2. Make a paid plugin (but placing into main repository it is available to everyone open to buy it)
  3. Install you plugin manually

First two option are the most widely adopted by public plugin sellers, while the third one is a very common practice into agency or companies that have some common plugin reused in many websites.

WpGet came to give a fourth opportunity. It is conceived as a private repository where you can push plugins and allow wordpress installation to update from that source, in addition to the standard platform. For who comes from other technology WpGet is similar to NuGet, Maven or NPM package manager.

Here a simple diagram to explain how WGet works.

 

 

All code i hosted on git-hub where you can also download last release.

 

3. Functional requirements: What a public wordpress repository should be

In my experience I played with many package repository (maven with java, composer with php, nuget with .net and so on…) and I think in 2018 everybody use them each day during development. However what about wordpress plugin? In fact a plugin is just a zip file. A zip file with php file that implements features basing on wordpress standard, but simply a zip file from an higher view. So a plugin repository can be treated as a a raw zip file repository. Such purpose can be achieved using some ready product like nexus that can store versioned zip file into raw repository. While this could be ok for archiving plugin it may find some limits in the integration with wordpress itself because there isn’t any plugin that integrates such system. Searching into github there are some wordpress project that manage plugin using some http repository but no one seems to be so mature to be considered as a standard and no one has an UI that let you understand what happens behind the box.

 

To be more clear, what we need from this project is :

  • Provide updates for plugins, keeping all packages and providing to the wordpress installation. Using token based authentication all process are kept secure.
  • Easy to setup: Setup must be easy. Just a copy of the bundle to the server and few steps more. Plugin integration is very easy, just a class to add.
  • Api oriented All core feature are exposed by API. This will allow you to integrate the delivery process with other tools in the company.
  • Ui to get control: A simple ui show what package are loaded and give you the power.
  • Low Server Requirements Just a webserver with php and a database. For few data, using SQLite, neither database server.
  • Versioning: have a system that stores and version wordpress plugins
  • Integrable in development process:be able to publish wordpress plugin using a push script
  • Made for humans:simply upload packages manually if you don’t like shell commands
  • multi tenant: system that allow to manage multiple repository with same installation instance (i.e. one repository for each customer)
  • Secure: a secure system that allow only to trusted website to download packages



 

4. Technical requirements: How a private repository for wordpress should be done

The principles that drives  my design was:

  • employ standard technology
  • api oriented application
  • Backend\Frontend complete separation, SPA for the frontend
  • token based authentication
  • use technologies that can be compatible with wordpress to easy installation
  • to be delivered in a one-click installation package

 

To grant the compatibility of technology my choice for the backend was a PHP application based on Slim Framework. This is because I would use  a simple and lightweight framework as the application is designed to manage just few database entities and a very simple business logic. By the way Slim has a lot of modules that can be integrated, so further expansion of the application will be managed without any limits. Frontend is implemented using an angular application that is a powerful solution.

 

 

5. The backend

5.1. Multiple backend applications

WpGet is mainly designed for small applications like the web agency that develop some plugin and want to provide to their customer. Basing on such scenarios, is easy to image the load of application won’t be so heavy. Users don’t really need to play with ui all times, publish of package are not so frequent (1-2 per day), and clients will update wordpress installation only if there is changes in  plugins. We don’t have crystal ball to tell for sure, but probably performance and load won’t be an issues on most applicataction of WpGet. Anyway, I wanted to design an architecture that may scale in future, in example if we would to publish this application in SaaS environment. While this application don’t use sessions or other shared memory than database and storage folder it will be quite easy to proceed with horizontal scaling as users will grow. By the way, i also wanted to create a structure that allow to let application grow following application module usage. So, i splitted the monolithe into three components:

  • Auth: application that provide authentication
  • Api: application that hosts all api for UI and CRUD operations
  • catalog: application that orchestrate integration with wordpress and publishers

 

This will allow to destinate the right amount of resources to the single component. Moreover we will have three smaller application, that have only the required dependency each one, making them faster and easy to deploy. In the regular on-premise bundle all the application are bundled together to simplify installation, but, playing with configuration and servers we can simply move to the complex solution.

5.2. Automate routing

Slim framework is a great solution and allow quickly to produce a rest endpoint in a quick and dirty way.

You can use lambda in main application file or create invokable classes

Examples:

$app->any('/mycontrollerpath', function ($request, $response, $args) {
    // do stuff here
});
$app->any('/user', 'MyRestfulController');
class DynamicController
{
    public function __invoke($request, $response, $args)
    {
      // do stuff here 
    }
}

 

Meanwhile I resigned myself to work with untyped data delivered by service without a strong contract (like WCF or Web API, to explain better…), I wanted to avoid to enumerate all possible route into main application files. The best solution I imaged was to declare some classes or annotation, then decorate controllers, than make a scan at runtime and dynamically use such informations. This will had lead to something very similar to Asp.NET WebApi engine, but i preferred to found a simpler solution because:

  1. this is a very out-of-standard solution
  2. PHP doesn’t have any “native” application memory state so I had to add some memory state (i.e using memcache) or load each time wasting resources.

 

The solution I found was simple and very efficient:

  1. I create a base controller class, called DynamicController. This controller implement Invoke method and will receive all action calls, dispatching to a method with same name of the action and same http method. This is based on naming convention so you will have /mycontroller/myaction get call managed by getMyAction method inside Mycontroller class.
  2. The real implementation of action methods is inside a class that implements DynamicController.
  3. inside application file you have to add only one entry per controller

 

$app->any('/repository/{action}[/{id}]', RepositoryController::class);

 

5.3. Automate database creation

Another part I would improve is about CRUD operations. I used eloquent ORM and it is a very simple framework to set up. In it’s simple implementation to create a rest service that expose crud operation you need:

 

  1. Create a model class. The only required information is the name of the table.
  2. All access to fields can be considered by-name as php members are dynamic, so if you need to set\get a filed, just use it
  3. In query expression there is a fluent syntax that takes field names in string format
  4. You can use some API to manage schema operations.

 

In a simple scenario I could just used some db script to manage schema definition, but in this case, as the application will be installed using many different databases I want to set up a system that is not dependent on database dialect. So I declined the SQL option and start using eloquent APIs. Question is that I don’t want also to create an huge script that replicate same boring code that check if table exists, then create the field and so on. My solution was to:

  1. create an interface that represent the table and that ask to implementer to define basic informations (table name and field list, for example)
  2. I implemented one instance for each table telling what field I want in each table
  3. I created a Manager class that load all table classes, then apply schema changes basing on class definition

Example of interface

 abstract class TableBase
{
    public abstract function getFieldDefinition();
    public abstract function getTableName();

    //field list
    public function getBaseFields()
    {
        return array(
            'id' =>'bigIncrements',
            'created_at'=>'timestamp',
            'updated_at'=>'timestamp',
        );
    }

    function getAllColumns()
    {
        return array_merge($this->getBaseFields(),$this->getFieldDefinition());
    }

}

 

Example of implementation

 class UsersTable extends TableBase
{
    public function  getFieldDefinition()
    { 
        return array(
            'username' =>'string',
            'password' =>'string',
            'token'=>'string',
        );
    }
    
    public function getTableName()
    {
        return "user";
    }
}

Example: manager usage

 $um= new UpdateManager($dbSettings);

$um->addTable(new RepositoryTable());
$um->addTable(new PublishTokenTable());
$um->addTable(new UsersTable());
$um->addTable(new PackageTable());

$um->run();

 

5.4. Automate CRUD operation

As routing and ORM layer are setted up, I created an EntityController that’s an abstract, generic class that manage basic CRUD operation on generic model. To add an entity you just have to create a concrete class based on it an register the routing

 

Example: implement entity

 class User extends \Illuminate\Database\Eloquent\Model {  
  protected $table = 'user';
}

Example: implement controller

 class UserController extends EntityController
{
    public function getTableDefinition()
    {            
        return new UsersTable();
    }

     public function getModel()
     {
         return '\WpGet\Models\User';
     }
}

 

Example: register startup

 $app->any('/user/{action}[/{id}]', UserController::class);

 

All basic operation are already managed. In some case you can override the method and do additional business logic like special validation, compute field or save related entities.

5.5. Automate dependency management

If there is a things that stucks me using php is the dependency management. The concerns is not about composer and the package management that is awesome and resolved most of the problems. My complains is about the fact you need to link the files you will use into the main application using include statement or variants like include once, requires etc.. This is very annoying and may lead to issues when, like in our case, we will have many entry point (do you remember we decided to have three different applications?). In many php based solutions they implemented some bootstrapper to simplify the script loading (i.e. drupal). in this application i wanted to implement a simple bootstrap framework. The best approach I found in products is to define for each module or plugin the dependency list, then bootstrapper will collect and load all files basing on active plugins. In this case, as we do not have a plugin system but just three application I decide to avoid such overkill solution and found a simpler. I implemented a class “DependencyManager” that resolve dependency basing on a list of folders or files. Each application provide the list of dependency so that dependency manager will scan file system to load all php files.

 

Example: usage of dependency managment

$dm= new DependencyManager($appPath);

$dm->requireOnceFolders( array(
  'src/models',
  'src/controllers/base',
  'src/controllers',
  'src/db/base',
  'src/db',
  'src/handlers',
  'src/utils',
));

 

If application will grow in future and will need a real plugin system we will need to make the dependency manager read lists from database or some configuration files.

6. The Frontend

The UI application is written in Angular ( I mean the 2+ one, not AngularJS, if you was confused ;) ). Although I wanted to keep everything closer as much as possible to standards, there is some point that need some attention and be useful when you will design you next Angular application.

6.1. Ui framework: prime NG

The first question was about the component framework to use. My personal selection list:

  1. CoreUI
  2. PrimeNG
  3. Material

I think these are the best solution the market offers if you don't want to pay. From my experience I found material very hard to be managed and I prefer to not use if the context doesn’t require, despite it is a very good framework. CoreUI is very complete and I love it as you can produce nice application without have any graphic designer competence. Moreover the fact that is based on bootstrap helps a lot designer to look it better and customize. PrimeNG is not based on bootstrap and this brings some issues when I ask to a bootstrap guy to put hands on it. Nothing about PrimeNG framework, but just the question is that nowadays everybody already knows bootstrap, not all PrimeNg. By the way PrimeNG has the best grid for Angular 2+ so that's why in many project I used it mixed with CoreUI. By the way, in this case I wanted to keep things simple reducing as much as possible dependencies, so I simply used PrimeNG for all the application.

6.2. Implement http interceptor to manage authentication

This is a common issue: authentication client http calls to the server. Using token based authentication is easy to add a token into a request, question is do not do it each time. In angular I found two way to do this:

 

  • Extend HttpClient: in this way you extend the class, override methods and you can manage additional http header and redirection to login for not logged users.

  • Use interceptor: interceptor is like an hook that allow you to manipulate all request from the application in a single point. This wasn’t present at the beginning of Angular 2 but available since Angular 4.

 

There are some good reason to prefer HttpClient extension pattern in most application. The biggest problem is about how many HttpInterceptor definition may interact together and how to manage different external destination. In our application such issues are not so important as we send unauthenticated requests to the login and all other request are authenticate. By the way in a so simple application it was awesome to have a single point where manage all the stuff without asking  services to use the special http client instead of the base one. So I followed Interceptor way, here the basics to manage the token based authentication:

 

Example: code from interceptor

 @Injectable()
export class AuthInterceptor implements HttpInterceptor {

    forceLogout(): any {
        localStorage.clear();
        document.location.href= this.config.baseHref;
    } 

    intercept(req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {
  
      
       
        if( checkIfRequestNeedToken()) // checkIfRequestNeedToken code is omitted. In this case is the request to app backend
        {          
            //user null means no login done, user not null but 401 means token expired
            if (this.auth.currentUser()  ) {
             
                let url=req.url;
                return next.handle(req.clone({
                    setHeaders: {
                        'Authorization': 'Bearer '+this.auth.currentUser().token
                    }
                }))
                .catch((err, source) => {
                    if (err.status  == 401 || err.status  == 0) {
                        this.forceLogout();
                           return Observable.empty();
                       } else {
                           return Observable.throw(err);
                   }  
                   });
            }
            else 
            {
                //completely force logout
                this.forceLogout();
                return;
            }
        }
        else
        {
            return next.handle(req);
        }
    }
}

6.3. Dynamically compute baseHref

BaseHref is a parameter that  tells to Angular the path relative from hostname and is required from routing. In standard application is easy to add a parameter to the build command basing on environment where you want to publish. This is quite easy to manage and doesn’t give you any real problem as you have basically many installation (prod, QA,test, integration) that are copies of the same. In this case, as application is an on-premise installation I can’t know where application will be installed. It may be in the root of the web site or in a subfolder. So I decided to hook baseHref basing on customer environment settings. This path is taken from a dynamic configuration variable, set during the installation. So, when the user will install, backend compute and store baseHref that is used from UI during regular usage. Here is the piece of code that setup the parameter. Please note that you could place here any logic to define your required values:

 

Example: base href override (app,module.ts)

 export function baseHrefFactory (config: ConfigurationService)  {
  return window.location.href.substring(window.location.href.lastIndexOf(config.baseHref));
}
//...
 providers: [
 //...
{
    provide: APP_INITIALIZER,
    useFactory:configFactory,
    deps: [ConfigurationService,HttpClientModule],
    multi: true
},
{ provide: APP_BASE_HREF, 
  useFactory: baseHrefFactory,
  deps: [ConfigurationService,HttpClientModule],
 },
 //...

 

6.4. Decouple configuration from compilation

Usage of environments is very useful and it is awesome that this development framework takes in account the possibility that an application will be deployed in more than one place. The matter is about the way it does it: at compilation times. While in a simple project you can do many build for each commit, producing different artifact for each deployment environment or rebuild code during release, I cannot build application for all the installation user will be done from users. So I decide to change this approach and  use dynamic settings.

Settings file is a json file placed into assets folder. For who will tell is not secure, I will remember also environment files are stored inside js file in plain text so it is the same security level. Information inside js are create at installation time and values are computed basing on the environment. This will allow us to make only one build, produce an universal package, and tune parameters during startup.

 

7. The wordpress plugin

About wordpress plugin it is needed to do a deep explanation about standard, core feature of the engine in order to explain it. This is an huge examination and a little bit off-topic from this article, that’s mostly focused on WpGet application. That’s why I decided to explain here only an high-level overview from the user side. The purpose of this chapter is explain the basics to create a new plugin compliant with WpGet specification. I’ll dedicate a dedicated article to the plugin architecture and about wordpress specific issues.

 

As already told a wordpress plugin is basically a zip file. Main idea is to put inside it a file in yaml format to explain what the plugin is. I chose Yaml format because it is a format easy to read and write and more suitable to contains textual data than XML or JSON format (just think about escape of special characters…) This manifest must have .wpget.yml name and during upload of package is read from application, no matter where it is. By the way it is recommended to place inside the plugin directory. Note that the yml file is placed inside plugin and can be used for the plugin itself to determine the current version and to display settings. What I described until now is how a wordpress developer can describe its plugin and load information into wpget system. Other question is how make a plugin able to interact with wpget server to get updatest. As ever there are many way to do this. During analysis I covered many of them :

 

  1. Manually add and register a class. We can provide to the user a php class, already working, that can be placed into plugin itself, registered and configured to make plugin updatable. This is the easier solution on vendor side (just prepare the class and a sample module), by the way is the less scalable as each user’s plugin must copy inside the class replicating the code and each plugin need some manual interaction.

  2. Create an umbrella plugin: This solution will pass the limitation of previous implementation by creating and releasing a plugin into public wordpress repository. This plugin will introduce in wordpress the basic classes needed for integration, so no more copy and past of the class. Moreover this will allow to manage basic settings for modules ( url of wpget, token for authentication) avoiding hard coded configurations.

  3. Create an alternative plugin manager: This will allow you to connect with many wpget repository, download full list of available packages then install\keep update them.

 

I decided to start from the simple solution for the vendor as the effort requested to the user is minimum. After this validation phase, if many user will report installation, I’ll consider to implement the solution (2) to give them an easier management on the wordpress side. About solution (3) i think it haven't any real advantages than (2) but it is in fact a too  much invasive implementation.

 

Coming back to the actual solution, we need two step. First you need to add the .wpget.yml file on the root, with this content

 

Example : yaml sample

 version: 3.7.0
name: my-plugin
homepage: https://github.com/zeppaman/WpGet
upgrade_notice: >
 [Improvement] New changes made in version 4.0 were causing problem at websites running on PHP version less than 5.0
author: Francesco Minà
author_profile: https://github.com/zeppaman/WpGet
requires: 4.9.4

Other parameter are omitted, you must check for complete yaml sample attacched to this article

 

Then you have to insert the php class (you can download here) and integrate into plugin file as follow:

 

Example : integration sample

// remember to set variables (es in wp-config.php file) before use this file

// // repository config data
// define( 'WPGET_REPO_SLUG','test000' );
// define( 'WPGET_PACKAGE_NAME','my-plugin' );
// define( 'WPGET_API_URL','http://localhost:3000/web/' );
// define( 'WPGET_TOKEN_READ','FnrvNuzwKodEgIqxsBctbFc2SxMncM');

// // plugin info
// // plugin filename
// define( 'WPGET_PLUGIN_FILE', 'plugin-test.php' );
// // plugin directory
// define( 'WPGET_PLUGIN_DIR', 'plugin-test' );

require_once( 'WpGetUpdater.php' );


 

8. Point of interest

In previous chapter I explored most interesting point of principal modules of the application. Moreover there are some important topics to discuss not directly related with frontend or backend that I will cover here. I’m referring to the DevOps part and to the packaging\installation process.

8.1. Installation process

As this application will be installed from user I would create an install procedure that will do all the required operations for you. This script will:

 

  1. check if application is already installed. If yes, simply do nothing
  2. ensure all folder are present with right permission
  3. create data schema in database
  4. produce settings file for frontend
  5. fail if one of previous step fail

 

Even if it would be beautiful this is an installation script, not an installation wizard. Thi means that user must follow this steps:

 

  • get a server
  • extract zip file somewhere on server disk
  • make the /web folder reachable to the web
  • insert database settings into settings file
  • run install script

 

To avoid to run non configured application i add a rewrite rule into .htacces (for apache installation) that redirect all traffic to the install page if application is not installed. This prevent that user that stop to read the previous dotted list and stop at point (2) get confused about what to do. Installation script check for write permission and helps a lot the user to reach the right server configuration. There isn’t really much settings to do except insert database connection settings, have some directory writable (storage, logs, assets) but I my experience I found better an application that doesn’t start until all is fine than one that doesn't work.

 

8.2. Build and packaging

This is an opensource project so code is hosted into github (for opensource code there are some alternatives nowadays?). This give me some interesting opportunities for automate the process, finally I choose circleci that’s free for public project and very easy to use. My build process simply download the code, deleted unused folders, build angular application. This is very easy but circleci run all the process in a docker image. He provide a node image and a php one, but no one with both of them together. This limit can be passed by creating a new docker image with node and php installed. I chose the lazy solution: I use the php docker image provided by circleci, but I installed node as first step of compilation. This will help me a lot in packaging phase as I have all files inside same folder, so I just need to produce a zip and upload to circleci as a zip file. This is the configuration for circleci with the installation script.

 

Example: circle ci config

# PHP CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-php/ for more details
#
version: 2
jobs:
  build:
    docker:
      - image: circleci/php:7.1.5-browsers
      
    working_directory: ~/repo

    steps:
      - checkout
     
      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "composer.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-
      - run: chmod -R 777 *
      - run: sh .circleci/build.sh
            
      - store_artifacts:
          path: /tmp/artifacts
      
      - save_cache:
          paths:
            - ./vendor
          key: v1-dependencies-{{ checksum "composer.json" }}
        
      # run tests!
     # - run: phpunitconfig.yml

Example: circle ci startup script

#install node
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php composer-setup.php
php -r "unlink('composer-setup.php');"
php composer.phar self-update
sudo mv composer.phar /usr/local/bin/composer
composer install -n --prefer-dist
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs

#angular build
cd src-ui;
npm install;
npm run ng build --env prod;
cd ..;

#delete unused folders
rm -rf src-ui;
rm -rf .circleci;
rm -rf .git;
rm -rf .vscode; 
rm -rf web/ui/assets/settings.json

#build artifact
mkdir /tmp/artifacts;
zip -r /tmp/artifacts/web.zip .

 

9. Conclusions

Developing this application give me the opportunity to test all the components of a modern web application with the stack based on

  • PHP
  • Slim
  • Angular

Apply such kind of application in the product on premise world bring out some issues that usually we do not meet in the development process for regular business applications. This is interesting because the experience made on this project can be used as basics for new projects or simply used if application requirements needs it.

 

10 References & History

References

  • Wordpress statistic usage https://w3techs.com/
  • GitHub Project
  • GitHub Plugin Projecty

History

  • Update binary, updated link : 2018-02-15
  • First release of this article: 2018-02-14
  • First release of  wpget: 2018-02-10

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Daniele Fontani
Chief Technology Officer
Italy Italy
I'm senior developer and architect specialized on portals, intranets, and others business applications. Particularly interested in Agile developing and open source projects, I worked on some of this as project manager and developer.

My programming experience include:

Frameworks \Technlogies: .NET Framework (C# & VB), ASP.NET, Java, php
Client languages:XML, HTML, CSS, JavaScript, angular.js, jQuery
Platforms:Sharepoint,Liferay, Drupal
Databases: MSSQL, ORACLE, MYSQL, Postgres

You may also be interested in...

Pro

Comments and Discussions

 
-- There are no messages in this forum --
Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web05 | 2.8.190214.1 | Last Updated 14 Feb 2018
Article Copyright 2018 by Daniele Fontani
Everything else Copyright © CodeProject, 1999-2019
Layout: fixed | fluid