Actions

LimeSurvey architecture: Difference between revisions

From LimeSurvey Manual
 
(77 intermediate revisions by the same user not shown)
Line 4: Line 4:


todo, constraints
todo, constraints
This is the architectural documentation for the LimeSurvey application. It does not document the ecosystem surrounding it.
The architectural decision roundtrip should look like this:
[[File:arch_roundtrip.png]]
This document should be reviewed, updated and discussed at least yearly.


== Vision ==
== Vision ==


todo
Stable software
 
Easy to change
 
Up to date with modern paradigms


== Technology stack ==
== Technology stack ==
Line 22: Line 34:
* Yii 1.1
* Yii 1.1
* jQuery
* jQuery
* Vue 1
* Vue 2 (to be removed)
* PHPUnit
* SASS
* React


'''Tools:'''
'''Tools:'''


* Git
* Git  
* GitHub Actions
* GitHub Actions
* Psalm
* Psalm
Line 34: Line 47:
* Scrutinizer
* Scrutinizer
* Selenium, geckodriver
* Selenium, geckodriver
* Yarn
* PHPUnit, Mockery
* Jest (JavaScript unit testing)
 
''' Dependency management:'''
 
* Manually (should be refactored)
* Yarn 1
* npm
* Rollup
* Composer


'''Libraries:'''
'''Libraries:'''
Line 45: Line 67:
* PHPMailer
* PHPMailer
* pclzip
* pclzip
'''PHP extensions'''
* SimpleXML
* ZIP
* GD
* Sodium


== Risks ==
== Risks ==
Line 51: Line 80:


Risk identification, analysis, mitigation, monitor.
Risk identification, analysis, mitigation, monitor.
todo, probability, impact, supporting knowledge
todo, security risks
For common lists of risks, see Pandian<ref>Applied Software Risk Management, Ravindranath Pandian, 2007</ref>


=== Product risk ===
=== Product risk ===
Line 56: Line 91:
These risks are considering the individual LimeSurvey user or admin.
These risks are considering the individual LimeSurvey user or admin.


* Data loss (known or unknown)
Possible mitigations are in sub-lists.
* Unauthorized data access
 
* Blocking bugs (that leads to data loss)
{|
! Risk || Probability || Impact || Mitigation
|-
| Data loss (known)
| Low
| Medium to high
| Unit-tests and integrity tests in the CI
|-
| || || || Internal QA culture
|-
| || || || Possibility to enable extensive logging, from which you can reconstruct answers
|-
| || || || Regular backups
|-
| Data loss (unknown)
|
|
| ?
|-
| Unauthorized data access
|
|
| Full test-suite of the permission system
|-
| Blocking bugs (that could lead to data loss)'
|
|
| Full test-suite of the high-priority functionality
|-
| || || || Full manual QA testing before each major and minor release (not for patch releases)
|-
| Low user experience (UX) || High || Medium to high || Regular feedback surveys from users
|-
| || || || Regular reviews by professional UX designer
|-
| || || || High changeability in the frontend of the product
|}


=== Project risk ===
=== Project risk ===


todo
todo, pull request process


* Communication
{|
! Risk || Probability || Impact || Mitigation
|-
| Communication || Medium || Low to medium || Weekly meetings
|-
| || || || Yearly LimeCon
|-
| || || || Virtual LimeCon
|-
| Regressions || High || Medium to high || Functional testing for each new feature
|-
| || || || Continuous effort to expand the test suit
|-
| || || || Effort to keep an eye on incoming bug reports
|-
| || || || Effort to create bug statistics and improve bug statistics
|}


=== Business risk ===
=== Business risk ===


Internal
Internal to LimeSurvey GmbH


== Quality attributes ==
== Quality attributes ==


Maintainability, easy to deploy, testability
Also called non-functional requirements.


== Technology stack ==
Considering the risks above, the most important quality attributes could be considered:
 
* Testability
* Changeability
* Usability, or user experience (UX)
 
They can be improved by using dependency injection, and having clearer separation between the abstraction layers, including splitting the monolith into separate modules.
 
Function composition and the "functional core, imperative shell" pattern could also help.
 
'''Strategies to improve testability:'''
 
* Explicit dependencies
* "Functional core, imperative shell"
* Make testability explicit when designing new features
* Make testability part of refactoring efforts
 
'''Strategies to improve changeability:'''
 
* Refactor functions into smaller parts
* Improve documentation
* Improve and extend the tools for continuous integration (Psalm, code duplication, metrics)
* Factor out widgets and/or JS components
 
To improve changeability, the system must become less fragile. This can only happen with good separation of concerns and a CI.
 
Code should consist of composable and interchangeable parts.


Dependency management, packaging, plugin
'''Quality attributes that have good performance:'''


todo
* Performance and scale
* Availability
* Easy to deploy
* Security (but requires continuous and conscious efforts)


== Views ==
== Views ==


4+1 view model?
Using the 4+1 view model from Kruchten<ref>Architectural Blueprints - The "4+1" View Model of Software Architecture</ref>, you can split the documentation into five parts:
 
* Logic
* Development
* Process
* Physical
* Scenario
 
Loosely interpreted, for us is will be:
 
* Domain diagram
* Class and layer diagram
* Flow diagrams
* Deployment diagrams
* Scenarios (possible omitted)


=== Domain ===
=== Domain ===
Line 100: Line 230:
* Business logic layer, calculating business rules, should be fully unit-testable
* Business logic layer, calculating business rules, should be fully unit-testable
* Database storage layer (also includes domain entity validation)
* Database storage layer (also includes domain entity validation)
Refactoring should be focused on strengthening the separation between the layers.


[[File:ls_arch.png]]
[[File:ls_arch.png]]


[https://github.com/olleharstedt/diagrams.net/blob/main/LS%20Arch.drawio Source].
[https://github.com/olleharstedt/diagrams.net/blob/main/LS%20Arch.drawio Source].
Overview of future refactoring efforts.
[[File:ls_arch_refactor.png]]
Forbidden transgression between abstraction layers:
[[File:ls_arch_forbidden.png]]
How splitting up a monolith into separate modules could look like:
[[File:ls_arch_modules.png]]
Possible architecture of REST + SPA module:
[[File:ls_arch_rest_spa.png]]
todo, mobile app architecture
=== Class hierarchy ===
Model hierarchy, major groups:
* LSActiveRecord - base class
* Localization tables, L10n
* Survey tables - survey, quota
* Group tables
* Question tables - subquestion, answers, default answers, labels, condition, timing
* Participator tables - token, custom attributes, CPDB
* Plugin tables
* User tables
* Themes - question themes, survey themes, configuration
* Admin customization - menus, boxes, notification, tutorial
* Permission tables - user groups, permission
[[File:arch_models.png]]


=== Flow and state ===
=== Flow and state ===


automata between states
Most basic flow when administer a survey.


admin flow diagram? what makes sense to document? create survey process?
[[File:arch_survey_flow.png]]
 
Overview over the survey taking state machine.
 
[[File:arch_survey_taking_flow.png]]
 
=== Deployment ===
 
Server, PHP-FPM, MySQL, browser, LimeSurvey installation


== API design ==
== API design ==
Line 118: Line 294:


todo
todo
Permission system
Users
User groups
== References ==
<references/>

Latest revision as of 16:31, 5 December 2022

This is the main public document for LimeSurvey's architecture

Overview

todo, constraints

This is the architectural documentation for the LimeSurvey application. It does not document the ecosystem surrounding it.

The architectural decision roundtrip should look like this:

This document should be reviewed, updated and discussed at least yearly.

Vision

Stable software

Easy to change

Up to date with modern paradigms

Technology stack

Programming languages:

  • PHP 7.2+
  • JavaScript ES6 (admin) and IE11 compatible (survey taking)
  • CSS, HTML
  • XML, JSON

Frameworks:

  • Yii 1.1
  • jQuery
  • Vue 2 (to be removed)
  • SASS
  • React

Tools:

  • Git
  • GitHub Actions
  • Psalm
  • CodeSniffer
  • MessDetector
  • Scrutinizer
  • Selenium, geckodriver
  • PHPUnit, Mockery
  • Jest (JavaScript unit testing)

Dependency management:

  • Manually (should be refactored)
  • Yarn 1
  • npm
  • Rollup
  • Composer

Libraries:

  • CKEditor
  • kcfinder
  • PDF?
  • Sodium
  • Twig
  • PHPMailer
  • pclzip

PHP extensions

  • SimpleXML
  • ZIP
  • GD
  • Sodium

Risks

As defined in the code quality guide.

Risk identification, analysis, mitigation, monitor.

todo, probability, impact, supporting knowledge

todo, security risks

For common lists of risks, see Pandian[1]

Product risk

These risks are considering the individual LimeSurvey user or admin.

Possible mitigations are in sub-lists.

Risk Probability Impact Mitigation
Data loss (known) Low Medium to high Unit-tests and integrity tests in the CI
Internal QA culture
Possibility to enable extensive logging, from which you can reconstruct answers
Regular backups
Data loss (unknown) ?
Unauthorized data access Full test-suite of the permission system
Blocking bugs (that could lead to data loss)' Full test-suite of the high-priority functionality
Full manual QA testing before each major and minor release (not for patch releases)
Low user experience (UX) High Medium to high Regular feedback surveys from users
Regular reviews by professional UX designer
High changeability in the frontend of the product

Project risk

todo, pull request process

Risk Probability Impact Mitigation
Communication Medium Low to medium Weekly meetings
Yearly LimeCon
Virtual LimeCon
Regressions High Medium to high Functional testing for each new feature
Continuous effort to expand the test suit
Effort to keep an eye on incoming bug reports
Effort to create bug statistics and improve bug statistics

Business risk

Internal to LimeSurvey GmbH

Quality attributes

Also called non-functional requirements.

Considering the risks above, the most important quality attributes could be considered:

  • Testability
  • Changeability
  • Usability, or user experience (UX)

They can be improved by using dependency injection, and having clearer separation between the abstraction layers, including splitting the monolith into separate modules.

Function composition and the "functional core, imperative shell" pattern could also help.

Strategies to improve testability:

  • Explicit dependencies
  • "Functional core, imperative shell"
  • Make testability explicit when designing new features
  • Make testability part of refactoring efforts

Strategies to improve changeability:

  • Refactor functions into smaller parts
  • Improve documentation
  • Improve and extend the tools for continuous integration (Psalm, code duplication, metrics)
  • Factor out widgets and/or JS components

To improve changeability, the system must become less fragile. This can only happen with good separation of concerns and a CI.

Code should consist of composable and interchangeable parts.

Quality attributes that have good performance:

  • Performance and scale
  • Availability
  • Easy to deploy
  • Security (but requires continuous and conscious efforts)

Views

Using the 4+1 view model from Kruchten[2], you can split the documentation into five parts:

  • Logic
  • Development
  • Process
  • Physical
  • Scenario

Loosely interpreted, for us is will be:

  • Domain diagram
  • Class and layer diagram
  • Flow diagrams
  • Deployment diagrams
  • Scenarios (possible omitted)

Domain

A simple domain diagram over LimeSurvey:

Source.

Layers

LimeSurvey uses a basic, monolithic MVC layered architecture.

  • View layer, serving HTML, JSON or XML to the calling part (can be browser or local API machine)
  • Controller layer, gluing parts together from the other layers
  • Business logic layer, calculating business rules, should be fully unit-testable
  • Database storage layer (also includes domain entity validation)

Refactoring should be focused on strengthening the separation between the layers.

Source.

Overview of future refactoring efforts.

Forbidden transgression between abstraction layers:

How splitting up a monolith into separate modules could look like:

Possible architecture of REST + SPA module:

todo, mobile app architecture

Class hierarchy

Model hierarchy, major groups:

  • LSActiveRecord - base class
  • Localization tables, L10n
  • Survey tables - survey, quota
  • Group tables
  • Question tables - subquestion, answers, default answers, labels, condition, timing
  • Participator tables - token, custom attributes, CPDB
  • Plugin tables
  • User tables
  • Themes - question themes, survey themes, configuration
  • Admin customization - menus, boxes, notification, tutorial
  • Permission tables - user groups, permission

Flow and state

Most basic flow when administer a survey.

Overview over the survey taking state machine.

Deployment

Server, PHP-FPM, MySQL, browser, LimeSurvey installation

API design

todo

Security

todo

Permission system

Users

User groups

References

  1. ↑ Applied Software Risk Management, Ravindranath Pandian, 2007
  2. ↑ Architectural Blueprints - The "4+1" View Model of Software Architecture
Retrieved from "http://manual.limesurvey.org/index.php?title=LimeSurvey_architecture&oldid=175215"