Planet Drupal

The state of website accessibility

Research conducted by WebAIM, a leading nonprofit organization based at the Institute for Disability Research, Policy, and Practice at Utah State University, found that 97.4% of home pages evaluated contained accessibility errors in 2021. You may be wondering how many users are affected by these errors. Put simply, the answer is a lot.

More than 14 percent of adults in the U.S. experience difficulty hearing and more than 16 percent have vision impairments, according to the CDC. Even before accounting for disabilities related to motor skills or cognition, this means tens of millions of people in the United States alone may be blocked from engaging with your content or finding the information they need on the internet. 

Additionally, people may experience a “temporary disability” due to environmental conditions (e.g. poor lighting). It is likely that as we age, all of us will experience some form of disability. Moreover, different devices and internet connection speeds can affect one’s ability to access content. Accessible website design truly benefits all of us and helps create a more equitable society.

WCAG compliance and the ADA

WCAG is developed by the World Wide Web consortium (W3C) in coordination with organizations all over the world. The goal of the guidelines, as stated by W3C, is to provide “a single shared standard for web content accessibility that meets the needs of individuals, organizations, and governments internationally.” 

WCAG documents provide a number of success criterions that a webpage must “pass” to be considered accessible. These criteria are broken down into four categories: perceivable, operable, understandable, and robust. There are currently two versions of WCAG (2.0 and 2.1) and three levels: A, AA, AAA. WCAG 2.2 is expected to be published before 2022 and will include all the elements of the previous versions, with new proposed success criteria added.

In the United States, digital accessibility is protected by the American Accessibilities Act, or ADA.  While there is not yet a federal law dictating a specific WCAG mandate for website ADA compliance, many lawsuits have been successfully fought against owners of web properties for failing to meet the needs of people with disabilities. For instance, lawsuits brought by the National Association of the Deaf against MIT and Harvard culminated in 2020 with both universities legally consenting to provide high-quality captions for all online video content. After the Covid-19 pandemic hit, lawsuits against educational institutions skyrocketed as it became clear that people with disabilities could not adequately access educational content online and that the issue was urgent.

What makes websites accessible

Website accessibility revolves around four user-centered principles: they must be perceivable, operable, understandable and robust. These principles are sometimes referred to using the acronym POUR. Each principle is explained below, along with examples of how the principle applies to website design and development.

Perceivable

The “perceivable” principle states that information and user interface components should be presented to users in ways that they can perceive. For instance, the criterion that text alternatives must be provided for any non-text content falls under this rule. You may already be aware that most images need to be accompanied by alternative text content that accurately describes the content to users who cannot see the screen. Similarly, input fields must be labeled or named programmatically so that the field can be identified by a screen reader or other assistive technology.

The perceivable rule also dictates that color plays an important role in accessible website design. While color cannot be the only means of conveying information or indicating an action, minimum color contrast plays an important role in the visual presentation of text. Many websites use color schemes that aren’t perceivable by people with impaired vision, including many types of colorblindness. In order for most text to be considered compliant, it must have a contrast ratio of 4.5:1. Luckily, website designers can use a number of tools to determine exact color contrasts and develop effective, accessible color palettes.

Operable

In order for a website to be “operable,” its user interface and navigation components must be functional for all users. A large component of operability is keyboard accessibility. Content that is operable only through mouse clicks will not be usable for folks who cannot use a mouse. And while some folks may not be able to use a standard keyboard, most adaptive technologies emulate the keyboard. This makes keyboard accessibility important to provide access to people with motor disabilities, people who are blind, and people with conditions such as tremors.

The operability principle also applies to the ways that users find and navigate content on your website. Many people don’t realize the importance (or existence!) of semantic structure in websites. Semantic structure—implemented through HTML elements such as regions, headings, lists, and ARIA specifications—is essential to helping users with and without disabilities get the information they need from your website. It is particularly important to folks using assistive technologies, such as screen readers, so that they don’t have to comb through every word on the site to find the content they need.

Understandable

The “understandable” principle speaks to a user’s ability to understand the user interface and the information it contains. This principle includes elements of design, development, and the content itself. For instance, one component of understandability is readability. This includes publishing content at the lower secondary education level and providing mechanisms for identifying jargon words (or avoiding them altogether!) and abbreviations.

In addition to the readability of text, an interface must be predictable and offer input assistance to help users avoid and correct mistakes. This includes making sure elements that perform the same function are labeled consistently and appear in the same order, where relevant. For instance, a button labeled “Previous page” should not be labeled “Go back” in another location. These types of labels are most often implemented programmatically by developers as they create your website. Similarly, error prevention, such as letting a user know when a form input isn’t valid or a field, is required as part of what makes a website understandable.

Robust

To be considered technologically “robust,” an interface must be able to be interpreted reliably by a wide variety of “user agents” — mainly browsers and assistive technologies. One of the main elements of robustness is called parsing, which means that mark-up languages such as HTML must be formatted and nested according to proper specifications so that they can be properly analyzed and interpreted by these user agents. A simple extra quotation mark or forgotten angle bracket could make a web page programmatically unreadable.

Another important consideration for robustness is the implementation of status messages. This criterion states that status messages should alert users without interrupting the workflow of a user of assistive technology. For instance, if a user is scrolling through a social media feed when an alert message pops up, the user should be made aware of the alert without the focus (the input area of the screen that receives keyboard input) shifting.

Focus on accessibility, every step of the way

To ensure that websites meet the needs of all users and are WCAG compliant, it’s important to prioritize accessible website design every step of the way—from user experience research to content entry.  Here’s an overview of the type of accessibility work that happens throughout the web design and development process.

User experience research

User experience (UX) research is the study of end users and their requirements and wishes. While much user experience research is conducted before a project even begins, the UX project should be iterative — with a testing and feedback cycle that continues even after a website is launched. Accessibility should be taken into account from this early stage in identifying the needs of the users. This can be done by incorporating users with disabilities in the research, building accessibility considerations into personas and user stories, and testing prototypes at various stages using assistive technologies.

User experience design

The UX design process—to include information architecture (organization) and interaction design—can help head off accessibility concerns down the road by ensuring that simplicity, consistency, and user needs are the backbone of a website. At this stage, accessibility can be built in by establishing consistent navigation, metadata, and taxonomies, planning for consistent UI components to be used across a website, and determining the hierarchical and semantic structure of web page templates.

At this stage, designers also consider how to convey information and offer user input across different devices, develop alternatives to motion for alerts and emphasis (which can cause harm to people with certain conditions), and ensure that a meaningful linking strategy is employed to avoid the use of inaccessible “click here” links throughout a website.

Visual design

In the visual design stage, the perceivable principle takes center stage. Visual designers develop accessible color pallets, decide on typefaces and font sizes for various user interface elements, and ensure element selection and alerts are conveyed without color alone. 

Visual designers also play a role in consistency (an important feature of the understandable principle) by ensuring the consistent patterns built into the information architecture are conveyed graphically.

Development

The development stage is where the bulk of accessibility implementation happens. It’s at this stage that mark-up elements are created and formatted according to specifications with proper names and labels. CSS is used to implement accessible styles, responsive designs, and media queries, and create interactivity with scripting languages. Developers build in keyboard accessibility so that any interactivity, any action a user can take using a mouse, can also be performed with a keyboard or device that uses keyboard inputs.

Developers can also implement Web Accessibility Initiative – Accessible Rich Internet Applications, better known as WAI-ARIA or simply ARIA. This framework helps assistive technologies properly interpret interactive website components (e.g. sliders or accordions) by providing additional attributes.

Content entry

Entering accessible content into a content management system, such as Drupal,  is the final step. This includes ensuring images and non-text images have helpful alternative text, using headings in the proper order, and providing high-quality captions and/or transcripts for audio/video content. Learn more about writing accessible web content on our blog from one of our designers.

Feeling overwhelmed? Get in touch!

If you’re new to accessible website design or WCAG compliance, this might seem like a lot to keep in mind! At Redfin Solutions, we pride ourselves in meeting our clients where they are, and we’re here to help with important web considerations like accessibility. 

If you’d like an accessibility audit on an existing web property or are interested in starting an new accessible project, get in touch!

What to know about Drupal design

It's not (quite) a blank slate for designers

It’s true that Drupal is highly flexible. You can customize almost everything on your Drupal website, from layouts and colors to content organization and integrations. However, it’s important to keep in mind that as a content management system (CMS), Drupal relies on some standard “building blocks” to allow for quick and consistent content authoring. Every Drupal website has a Theme, or a collection of files that define the presentation of your website. In addition to standard assets like CSS and Javascript files, some of these are Drupal-specific files. For instance, Drupal regions must be added to a theme to control where the content is displayed and how the page is marked up in HTML. It’s necessary for Drupal web designers to understand the basic mechanics of a Drupal website and understand how design concepts will translate into Drupal development.

Design is component based

If you’re following recognized best practices for scalable web design and development, you’ll want to think about your Drupal design in terms of components rather than just pages. In practice, component-based design (also called Atomic Design) means that designs are broken down into smaller component parts that can be combined and rearranged in a number of ways to create different page templates. For example, your smallest components may include buttons, labels, and input boxes. These components always look the same, but they can be re-used and rearranged to create search bars, sign-up forms, or calls to action. These elements can then be combined with others to create larger elements such as headers or modals. 

While component-based design is popular across the web, it’s especially important for Drupal websites in order to take advantage of the flexibility and scalability Drupal offers while maintaining consistency and easy content authoring across a complex website. Drupal 8 has modules such as Layout Builder (included with the Drupal Core distribution) and Paragraphs that make implementing component-based design much easier than it’s been before.

Your information architecture matters

When you think of a website design, visual elements like colors, images, and typography probably come to mind. These visual components are undoubtedly important, but it’s likely that the information architecture underlying the visual design is just as important—if not more important—to helping users find content and maintaining an attractive, consistent, and user-friendly website. Some ways that information architecture is manifested include site maps, hierarchies and categorizations (i.e. taxonomies), navigation, and metadata.

Choosing the right Drupal web design agency

Now that you know the basics of what designing a Drupal website entails, how do you find the right agency for the job?

Look for Drupal design examples

Often, the most sure-fire way to see whether an agency has the Drupal design experience you’re looking for is to check out the other work they’ve done. Think about the different features of your web design project that are the highest priority or may be unique to your website and make sure the agency you hire has a track record of success on similar projects. For instance, if you’re redesigning a website for a university office, you may look for companies that have previously designed higher education websites. Or if your website needs a reservation system, check if the agency you choose has experience integrating websites with reservation software and constituent relationship manager (CRM)s. Of course, you’ll want to ensure the examples you see are from Drupal websites!

Ask about the design process

If you’re unsure about an agency’s design process or it’s not clear in their proposal, it’s more than okay to ask! The agency you choose should have an established “Discovery” phase devoted to understanding your business goals and your users. This might involve creating user personas or defining the key tasks you expect users to be able to accomplish on your website. You may also want to gain clarity about what design deliverables they’ll provide. Depending on the size and scope of your project, this might involve sitemaps, user journeys, wireframes, or design mocks, for instance. Most importantly, ensure that the design process is user-centered.
These days, it’s a good sign if an agency follows an agile approach to design—meaning work is conducted iteratively in two week increments called sprints. The Agile methodology allows stakeholders to provide more feedback and re-examine goals as the project moves along. You can also ask about how designers and developers at the agency work together. A good rapport and established workflow between designers and developers goes a long way toward ensuring your final product looks like the mocks a designer shows you.

Check out their involvement in the Drupal community

The best way to tell whether an agency has real Drupal credentials is to take a look at their involvement in the Drupal community. If they’ve contributed code to Drupal modules or have played a part in organizing Drupal events, it’s likely they’ve committed to keeping up with best practices for designing Drupal websites. At Redfin Solutions, for instance, we regularly sponsor Design 4 Drupal, Boston, an annual conference devoted to design, UX, and front-end development for Drupal websites.

Assess their communication style

Remember, you’re going to be working closely with the web design agency you choose. Beyond technical skills, it’s important that a web design agency prioritizes communication skills and customer service. Even the most genius tech wizards are only as good as their ability to communicate with you and listen. After all, it is your website.

Red flags: what to look out for

Don’t get caught off guard when it’s time to launch your website. Keep these red flags in mind when you’re choosing an agency to ensure a successful project.

Avoid designers who don't know Drupal

There are many great web designers out there that aren’t familiar with Drupal. You should avoid them if you’re building a Drupal website. This isn’t because their designs won’t look great on Drupal—it’s because there’s a good chance they won’t know how to implement them in a Drupal environment. This applies to any software or content management system. Choosing a web design agency that specializes in the platforms that your website uses will ensure you get the most out of the technology you’ve got.

Beware an inadequate discovery phase

In an attempt to provide you with the lowest quote possible, a design agency might nix or skimp on the discovery phase. But without this essential first step in the process, an agency can’t possibly build a user experience that delights your users and helps you achieve your business’s specific goals. When it comes to Drupal design, context is everything. Make sure the agency you choose for your project takes the time to understand the environment in which you operate rather than just churning out websites factory style.

Steer clear of agencies that don't mention accessibility

If a proposal from a web design agency doesn’t mention how they’re addressing web accessibility or usability, you should consider this a red flag. Following accessibility standards in design and development allows users with disabilities, such as visual impairment or limited mobility, to access your content—and helps everyone find the information they’re looking for. Depending on your website’s domain, it may even be legally required that your website meet WCAG criteria. When it comes to accessibility, there are no shortcuts.

Conclusion

Keep these tips in mind when you’re ready to find the right Drupal agency for you and your website. To learn more about user-centered web design for Drupal websites, visit our blog or contact us to see how Redfin can help!

So the Drupal 7 website you’ve been tasked with upgrading to Drupal 8 has WYSIWYG fields filled with various images, videos, iframes, and tables all with inconsistent formatting. You want to take advantage of this upgrade by switching to a more structured content editing system like Paragraphs, so all those special cases will have a consistent editor experience and display. There is too much content to do this manually, so you need an automated migration. But to impose all this structure, you need to intelligently divide ambiguous content into your specific destination paragraphs. This is a difficult migration task for a few reasons.

• There are multiple paragraph types, so they can’t share one migration.
• The original WYSIWYG content can be broken into several paragraphs of several of different types, but the Drupal migration API wants one source entity to become one destination entity as discussed in this blog post.
• The destination node needs to reference the paragraphs in the exact order as the original content.

These are tricky situations without a widely agreed upon solution. There are some custom migration modules that could help like Migrate HTML to Paragraphs. However, they may not fit your exact guidelines, especially if your paragraphs are referencing further entities like media or ECK entities. So how do you handle this?

There is no perfect solution. That said, one recipe for success is to write a custom process plugin that breaks up the WYSIWYG content, creates the correct paragraphs on the fly (as well as any file/media/ECK entities), and returns the paragraph IDs in the correct order for the destination node to reference. But beware: these paragraphs can’t be rolled back or updated through the migration pipeline. This makes testing more difficult because running a migration update is no longer idempotent, which means running it multiple times in a row will stuff the database with orphaned paragraphs. This simplifies the whole problem to one custom plugin. Here, the credit belongs to Benji Fisher for the starting point code. Keep in mind there are two shortcomings with this code that we will resolve later on.

Let’s break it all down. The first step is to import the WYSIWYG data into a DOMDocument in order to programmatically analyze the HTML. The DOMDocument is a data tree where each HTML tag is represented as a DOMNode that references any tags inside of it as children. You want to split up this DOMDocument so that each piece of content gets neatly mapped into the best fitting paragraph type. Since most of the content is simple text, the text paragraph type will be the default. So you need to test each piece of content to see if it matches a different paragraph type like image, video, or table. If it doesn’t match any of them, then you can safely drop it into the default text paragraph. You should start this process by iterating through all the top-level DOMNodes. For example:

<div class="wysiwyg-container">
  <div class="top-level">
    <p class="text"></p>
  </div>
  <div class="top-level">
    <img class="image" />
  </div>
  <div class="top-level">
    <p class="text"></p>
  </div>
  <div class="top-level">
    <table class="table"/>
  </div>
</div>

When you start with the wysiwyg-container div, you will see four direct children with the top-level class. Starting with the first child, test for each special case. One of the advantages of using a DOMDocument is that you can use the getElementsByTagName function to ask any DOMNode if it has children tags like img, table, or iframe. On a match, turn all the content in the branch to a new paragraph. Otherwise, put each chunk of consecutive text into a new text paragraph. Here lies the first problem with Benji’s template: branches with mixed content.

Most of the time this isn’t an issue. If there’s a table inside a div, you probably want the entire div for your table paragraph. However, if there’s an image inside an anchor tag, your image paragraph will grab the image data but ignore the link, losing data along the way.

{# Top level tag #}
<div class="top-level">
  {# Link around image #}
  <a>
    {# Image #}
    <img />
  </a>
</div>

This may be an acceptable loss, or an issue that can be flagged for manual intervention. Otherwise, you’ll need a method that can recursively traverse an entire branch of the DOMDocument, cherry-pick the desired HTML element, and put the rest into the default bucket.

For instance:

  /**
   * Recursively navigate DOM tree for a specific tag
   * 
   * @param $post
   *   The full page DOMDocument
   * @param $parent
   *   The parent DOM Node
   * @param $tag
   *   The tag name.
   * @param string $content
   *   The content string to append to
   *
   * @return []
   *   Return array of DOM nodes of type $tag
   */
static protected function recursiveTagFinder($post, $parent, $tag, &$current) {
  $tagChildren = [];
  // Iterate through direct children.
  foreach ($parent->childNodes as $child) {
    // DOMText objects represent leaves on the DOM tree
    // that can't be processed any further.
    if (get_class($child) == "DOMText") {
      $current .= $post->saveHTML($child);
      continue;
    }
    // If the child has descendents of $tag, recursively find them.
    if (!is_null($child->childNodes) 
      && $child->getElementsByTagName($tag)->length != 0) {
      $tagChildren += static::recursiveTagFinder($post, $child, $tag, $current);
    }
    // If the child is a desired tag, grab it.
    else if ($child->tagName == $tag) {
      $tagChildren[] = $child;
    }
    // Otherwise, convert the child to HTML and add it to the running text.
    else {
      $current .= $post->saveHTML($child);
    }
  }
  return $tagChildren;
}

If a top level DOMNode indicates that it has an img tag, then this method will search all the DOMNode children to find the img tag and push everything else into the default text paragraph, the $current variable. There will likely be some disjointed effects when pulling an element out of a nested situation: a table may lose special formatting or an image may not be clickable as a link anymore. Some issues can be fixed in the plugin. In the migration, I checked if any img tags had an adjacent div with the caption class and stored that in the caption field on the paragraph. Again, others may need to be manually adjusted, like reordering paragraphs or adjusting a table. Remember that it’s much faster to tweak flagged migration content than to find and fix missing data manually.

On to the next issue, let’s dig into embedded media in Drupal 7. The tricky aspect here is that the media embed is not stored in the database as an image tag, but as a JSON object inside the HTML. This requires a whole new set of tests to parse it out. To start, check each DOMNode for the opening brackets of the object [[{ and the closing brackets }]]. If it only contains one or the other, then there isn’t enough information to do anything. If it contains both, then get the substring from open to close and run json_decode. This will either return an array of data from the JSON object or null if it’s not valid JSON. The data in this array should contain an fid key that corresponds to the file ID of the embedded image. That file ID can then be used to grab the image from the migration database’s file_managed table and create an image paragraph.

Those are the main gaps in Benji Fisher’s custom source plugin. Of course each implementation requires even more tweaks and data manipulations to get the exact migration to work correctly. Some content simply will not transfer cleanly, so remember to test thoroughly and stay in tight communication with your client in order to turn WYSIWYG chaos into neat paragraphs.

If you found this migration series useful, share it with your friends, and don’t forget to tag us @redfinsolutions on Facebook, LinkedIn, and Twitter.

Migrating from Drupal 7 to Drupal 8 often requires restructuring your content, like transforming an unlimited text field into paragraphs or a list text field into a taxonomy reference. The tricky part is that the migration pipeline wants each source entity to go to one destination entity, but each paragraph or taxonomy term is a new entity and a single node can reference several of these.

So how do you break up data from one entity and migrate it into multiple entities?

Manual entry

If there’s a small set of old content or you’re already manually adding new content, then manual entry is a viable solution, but it shouldn’t be the default for large migrations. Going this route, you want to set up content editors for easy success. If possible, reduce the number of actions needed for a repeated task. With some clever string concatenation in your query results, you can create exact links to all the node edit pages that need updating. This is much easier than giving someone a node id or page title and asking them to fix that page. Just because it’s not an automatic migration, doesn’t mean we can’t automate aspects of it.

CSV Importer

The CSV Importer module is useful for simple data that already exists in a CSV file or can be quickly exported as a CSV file. For example, a spreadsheet with hundreds of country names could easily be imported as taxonomy terms with this tool. Or a list of emails and names could be imported as Users. Once you’ve migrated your data, you can reference them in other migrations using the static_map plugin or a custom process plugin to lookup the correct entity reference. Be careful not to abuse the static_map plugin with hundreds of mappings. In the country example, if the source data contains the name of the country that you want to reference in the destination, you could write a process plugin that gets the taxonomy id from the name. Remember that once entities are migrated you can use the full power of Drupal to find their id’s in later migrations.

Generate entities during migration

Use the entity_generate plugin or a custom plugin to create the entities during the migration process. This gives more control over how the data is transformed, but there’s no way to rollback or update the generated entities through the migration API. This shouldn’t be the default, but can be necessary for more complicated matters such as breaking down a dense wysiwyg field into separate paragraphs (see Benji Fisher’s custom process plugin).

Migrate entities separately with a custom source plugin

See our earlier blog post for a step-by-step guide on this. Drupal core provides lots of useful source plugins, but sometimes you need a custom query to migrate specific source data into entities. This approach gives you that flexibility within Drupal’s migration workflow. Unlike the previous option, you can still rollback and update entities and leverage all the other migration tools.

How you perform a data transformation like this is largely contextual, but these are powerful tools that can be used in many cases. Contact us with any questions regarding complex Drupal migrations, or if you are looking for a Drupal agency to help with your next website project.

This blog uses the 8.x version of the sqlsrv module. For Drupal 9+ implementations follow updated documentation from the module.

Most often in Drupal 8, your migration source will be a CSV file, JSON file, or another Drupal database. In some cases your Drupal website needs to coexist in a larger infrastructure. Thankfully, there are various modules for synchronizing Drupal with tools like Salesforce, Bynder, and GatherContent. However, not everything is as clean as those user-friendly modules. This article will dig into migrating data from a Microsoft server using Transact-SQL into a standard Drupal 8 website.

As with any database in Drupal, it starts in the settings.php file. The basic setup for a Transact-SQL database looks like this:

$databases['YOUR_DATABASE_NAME']['default'] = array (
  'database' => '',
  'username' => '',
  'password' => '',
  'prefix' => '',
  'host' => '',
  'port' => '',
  'namespace' => 'Drupal\\Driver\\Database\\sqlsrv',
  'driver' => 'sqlsrv',
);

“YOUR_DATABASE_NAME” is the key Drupal will use to reference this database, but it does not need to match the actual database name. The other credentials such as database, username, password, prefix, host, and port, need to be filled out based on your specific setup and server, but the last two keys are more general and refer to the type of database.
By default, Drupal uses a MySQL database, so the “driver” field is typically set to “mysql.” However, Drupal by itself does not know how to communicate with a Transact-SQL database, so just setting the “driver” to “sqlsrv” will throw an error

To provide that support, first install and enable the SQL Server module (sqlsrv). But the “drivers” folder in the SQL Server module needs to be accessed at the Drupal root folder (usually called “web” or “docroot”). There are two ways to do this:

1.  Manually copy the “drivers” folder from the SQL Server module (modules/contrib/sqlsrv/drivers) into the Drupal root folder.
2. Create a symbolic link (symlink) to the “drivers” folder from the Drupal root folder with a command like this “ln -s modules/contrib/sqlsrv/drivers/ drivers”. The symlink allows the module to update without manual adjustments. 

With the proper credentials and connections, Drupal will now be able to read from the Transact-SQL database.

Now the actual migration can be written. There is no core migration source plugin for this, so you will need to write a custom source plugin that extends DrupalSqlBase. Use Drupal’s dynamic query API to get the database's data, ensuring at least one field can be used as a unique identifier for each row. Once the source plugin is written, the rest of the migration will work as usual.

There are lots of tools built into Drupal 8 (D8) and Drupal 9 (D9) to assist with migrating from a Drupal 7 (D7) website. There is a whole suite of source and destination plugins that allow you to take data from any D7 node, file or user and migrate it into whatever D8 or D9 entity you want. But Drupal can’t account for every single data source you might have, so at some point in your migration you may hit a snag and need to write your own custom migration source plugin. Luckily, Drupal makes this straightforward.

If you don’t already have a custom migration module built, you can follow Mauricio Dinarte’s tutorial. Once you have that set up, go to your custom migration module and create the following nested folder structure for your custom source plugin:

your_custom_module/
├─ src/
│  ├─ Plugin/
│  │  ├─ migrate/
│  │  │  ├─ source/
│  │  │  │  ├─ CustomSourcePlugin.php

Then create a new PHP file in the source folder.

Now we can write our plugin. If you’ve never written a source plugin before, you can use the <a href="https://api.drupal.org/api/drupal/core%21modules%21node%21src%21Plugin%21migrate%21source%21d7%21Node.php/8.9.x">d7_node</a> source plugin for reference (this is the Drupal core source plugin for migrating nodes from a Drupal 7 database). Set up your namespace and underneath it add use Drupal\migrate_drupal\Plugin\migrate\source\DrupalSqlBase;. Then create a class that extends DrupalSqlBase with an @MigrateSource definition commented above it like so:

<?php

namespace Drupal\your_module\Plugin\migrate\source;
use Drupal\migrate_drupal\Plugin\migrate\source\DrupalSqlBase;

/**
 * 
 * @MigrateSource(
 *   id = "custom_source_plugin",
 * )
 */
class CustomSourcePlugin extends DrupalSqlBase {

Note that your class name should be in CamelCase as usual, but the ID should be in snake_case. The ID is how you will reference your source plugin in a migration YAML file.

Inside your custom source plugin, you will need to create four public functions: public function query(), public function initializeIterator(), public function fields(), and public function getIds(). <br/> <br/>

<strong>public function query()</strong>

Use this to query your source database (such as an old D7 website or whatever external database you are pulling data from). You will build your query using Drupal’s <a href="https://www.drupal.org/docs/8/api/database-api/dynamic-queries">Dynamic Query API</a>. In this example, I have a D7 website that uses an unlimited text field called field_footnote to add footnotes to a node. But in the new D8 website, I want each footnote to be its own entity, while making sure they each stay in the correct order on the correct page. This means I need to process footnotes one by one even though a single node can have several. I also need each footnote to know which node it came from and its order on the page, so the footnotes don’t get scrambled.

To handle this, the query is grabbing the “footnote” text field off all the nodes in the Drupal 7 source database with the entity_id, delta, and field_footnote_value fields selected:

$query = $this->select('field_data_field_footnote', 'f')->fields('f', [
  'entity_id',
  'delta',
   'field_footnote_value',
]);
$query->condition('entity_type', 'node', '=');
return $query;

This means that I process my old Drupal 7 field_data_field_footnote table row by row, turning each footnote into its own entity while keeping track of its parent (entity_id), order (delta) and value (field_footnote_value). This can be the trickiest step to get right because the whole source plugin depends on how the data is queried here. <br/> <br/>

<strong>public function initializeIterator()</strong> <br/> The initializeIterator function is necessary to run the query and start the iterator. All you need is:

$results = $this->query()->execute();
$results->setFetchMode(\PDO::FETCH_ASSOC);
return new \IteratorIterator($results);

If you need to set any constants for the migration, you can do that at this point as well: see <a href="https://api.drupal.org/api/drupal/core%21modules%21file%21src%21Plugin%21migrate%21source%21d7%21File.php/8.9.x">d7_file</a>. <br/> <br/>

<strong>public function fields()</strong> <br/> The fields function lets you state the queried fields from the source table and provide labels:

return [
  'entity_id' => $this->t('Entity ID'),
  'delta' => $this->t('Delta'),
  'field_footnote_value' => $this->t('Footnote'),
];

<strong>public function getIds()</strong> <br/> In the getIds function, you define which data field or fields will be used as unique identifiers. In my footnote example, multiple footnotes can be on the same node, so I need more than just the node ID to uniquely identify them. I also need to add the delta (the footnote’s order on the node).

return [
  'entity_id' => [
    'type' => 'integer',
  ],
  'delta' => [
    'type' => 'integer',
  ],
];

The entity_id and delta fields will get translated as sourceid1 and sourceid2 respectively in the migrate_map table. <br/> <br/>

That’s all you need to create your custom source plugin. Now you can write a migration to use it. Remember that in your migration YAML file you will use the @MigrateSource ID that you set in the comment above the class, not the class name.
 

If you want some extra credit, you can add a fifth public function, prepareRow. This allows you to make data manipulations on each row before it gets sent to the rest of the migration. If I wanted to set a character limit on my footnotes I could use prepareRow to trim or flag any footnotes that are too long. This can also be done as a hook in your module file, hook_migrate_prepare_row.

There’s a certain leniency afforded to migrating content from an old Drupal 7 website to its new Drupal 8 rebuild. Preserving old content with all its outliers can be complicated and tedious. However, it is typically run only once in production then augmented and completed. Take notes for future migrations and move on. This leniency is not available for ongoing migrations that need to be run routinely to keep a website’s content up to date. This can be anything from a website's directory to up-to-the-minute news and events data where the data comes from outside the website and requires a custom migration. Here the code needs to work consistently and report errors gracefully.
 

To run these migrations on cron, there are a few different options. The easiest is to use a module like Migrate Cron Scheduler or Migrate Cron. Both of these allow you to quickly pick a custom migration to run on cron and choose its frequency. Migrate Cron Scheduler, however, allows the user to configure the “update” and “sync” flags on the migration import.

The "update" flag will update all previously imported entities. The "sync" flag will also remove any previously imported entities that are no longer in the source database, so that the source and destination are synchronized.

These modules also let you choose a migration group, so that the migration import will automatically respect migration dependencies. The downside is that any custom code that needs to be run in conjunction with the migration will have to be split into a different cron job, and there is no setting on these modules to run the migration at a certain time of day or at an irregular frequency.

For situations that require these additional features, install and configure the Ultimate Cron module. This is a powerful tool that can be used to fine-tune any cron job to an exact time and frequency. The combination of Ultimate Cron and a migration cron module will cover the majority of use-cases. However, if custom code or reporting is necessary immediately before or after the migration runs, then a custom cron job may be needed. To build this, go to your custom migration module and create a hook_cron function in the .module file. This file will need to use the following classes:

use Drupal\migrate_tools\MigrateExecutable;
use Drupal\migrate\MigrateMessage;
use Drupal\migrate\Plugin\MigrationInterface;

Exactly how you configure this hook will depend on your situation. For each migration that you want to run on this cron job, you will need to follow these steps. First create an instance of the migration plugin by feeding the migration’s id into the Drupal plugin manager: $migration = \Drupal::service('plugin.manager.migration')->createInstance($migration_id);

Next, check the status of the migration. Often if there is an error, the migration will be hung in the “importing” state. Running a migration with the “importing” status will result in an error. To get around this, check the status and if necessary reset it to idle:

// Reset migration status.
if ($migration->getStatus() !== MigrationInterface::STATUS_IDLE) {
  $migration->setStatus(MigrationInterface::STATUS_IDLE);
}

At this point there are a couple options. You can leave the migration as-is and run a straight import where the content that has already been migrated will remain the same and any new content will be imported. Or set the “update” flag so that new content will be imported and existing content will be updated to match the source: $migration->getIdMap()->prepareUpdate();

Additionally, you can set the “sync” flag to remove any migrated content that no longer exists in the source database (e.g. an event that was cancelled). Note that this option requires at least migrate_tools version 5.

$migration->set('syncSource', TRUE);

Finally, create the MigrationExecutable plugin and run the migration:

$executable = new MigrateExecutable($migration, new MigrateMessage());
$status = $executable->import();

The “status” variable indicates whether the migration ran into any errors. If the status is empty, then there was an error and we can send a report and make custom adjustments to help the migration fail gracefully. Otherwise the migration ran fine.

Once the cron hook is ready, go to the Ultimate Cron jobs page (/admin/config/system/cron/jobs) and click the “Discover jobs” button. Your custom cron job should show up in the table, and you can click the “edit” button to fine-tune the exact timing. For updating or syncing migrations with hundreds of entities, the page may timeout if you manually hit the “run” button for your custom cron job. However, Ultimate cron version 2.x adds the drush commands cron-list and cron-run. Use cron-list to find your cron job id. Then use cron-run with that id and the “--force” flag to manually run your cron job for testing.

The Migrate Cron Scheduler or Migrate Cron module route is much faster to set up and is the better choice for quick, simple migrations that need to be run on the hour. However, for bigger, more complicated migrations that require additional code either immediately before or after the migration, or need to be run in the middle of the night, the Ultimate Cron module combined with a custom cron job provides the most customization and flexibility.

The content maximum width is the size at which the components (such as text or an image) on the website stop increasing in width but the backgrounds (such as a background color or image) keep going. If your content maximum width is particularly large, keep in mind how all your components look stretched to that width. For readability and accessibility of any text on the website, remember that optimal line length is 45-75 characters. 

This width may be all you need, but you can also define a website maximum width if you want something like an image background or full width slideshow to stop growing at a wider point. This may just apply to those full width components that are using images, or (less common in modern design) to the whole website. When deciding on whether or not full width components should keep stretching infinitely wide, think about how you want images to look on very large screens. Unless it’s a background image that is matching the height of that component, the wider an image gets, the more vertical space it will take. Also consider if you want large images to look clear at very large sizes or load quickly at normal and smaller sizes. Limiting the width an image component can reach is one way to strike a balance.

One last thing to define is the padding on the sides between tablet and desktop. When the width gets close to your content maximum width, how much padding stays on the sides as the content begins to shrink? I usually leave 16px between mobile and tablet, and 32px between tablet and the content maximum width (technically the content maximum width plus 64px so I don’t lose the padding too early).

Other common responsive considerations are typography and hover states. For typography, you’ll likely want to define heading sizes for mobile, tablet, and desktop. Also pay attention to line height, as well as spacing above and below a heading. For hover states, just remember that a mobile or tablet user can’t hover, so make sure none of the functionality is lost because of this.

Other tips

This covers most of the important points to keep in mind, but here are a few others:

Try to use a consistent base for sizing, preferably based on the body text size. I usually make my base font size 16px and then try to make typography and spacing stick with multiples of 4px.

Never type with caps lock. Always type in lower or sentence case and use your design software to transform the text to uppercase. This will help if you need to change that style without having to retype, and it allows developers (depending on your method of sharing designs) to copy and paste your text rather than retyping it. It also can be a good reminder to the developer that they should be using css to make text uppercase for accessibility (screen readers read text typed in all caps one letter at a time).

Make sure your images and text are flexible. If you have an image that perfectly lines up with the text overlaying it, be aware of how that component might be used and if the content editors will be changing the image or text. Similarly, be aware of the amount of text a content editor might add to any component. Character counts can always be limited, but it won’t hurt to supply that or show how the component should respond to having more text, especially if that text area is highly dependent on browser width.

One thing that can be easily overlooked is hover and focus states for any interactive elements of your website. For a website to be accessible, it has to be navigable with a keyboard. Focus states are what you see if you tab through a page, and if the difference isn’t notable it will be difficult to do this. Often, hover and focus states can be the same, but there may be some cases where you want to separate them. A form, for example, is an important place to have focus states defined, but you may not need hover states to work the same. Focus states that aren’t specified will generally be automatically provided by the browser, resulting in different appearances between browsers.

One last thing to establish while working with developers is a shared language. A paragraph or a block may mean something different to you as a designer then to a developer working in a content management system. Work together to make sure you have a common understanding of what some of these terms mean. A design system, or even a component library, is a great way to document this. We usually use Invision DSM to share a design system between designers and developers.

Summary

To review, remember the following things as you’re designing:

• Design mobile at 320px
• Define a content and website maximum width
• Create font styles for mobile, tablet, and desktop
• Use a consistent base for sizing, based on your body text size
• Get in the habit of using your UI to transform your text to uppercase
• Make images and text flexible whenever possible, and show components with varying amounts of content
• Provide hover and focus states
• Establish a shared language

When working with developers, it’s important to keep in mind the gaps between design and development when creating a website. While small design details may be obvious to you, a developer may be less aware of pixel distances and small color variations, focused instead on building the website in the best way possible. Ideally, design handoff is not one moment in a months long process, but a continual collaboration. Getting developer feedback while in the design process can save you from wasting time going down an unfeasible pathway, and getting across clear design patterns can ensure a developer makes the right assumptions from the beginning. Understanding a little of each other’s world can help make the whole process a lot smoother.