Monday, 17 July 2017

Use Regular Expression to parse the image reference in the markdown sourcre code

As mentioned in my blog A Github repository issue backup tool developed by ABAP I am developing a tool in ABAP which can backup my github issues.

One of the functionality I would like to fulfill is to backup the images used in issue with markdown format as well. Take this issue of mine for example. This issue contains the reference to image files uploaded by me via dragging my local image files and drop them to the browser.

The syntax is ![local file name, could be initial](image url), here below is an example:

Since I have already downloaded the source code of all github issues belonging to a given github repository into ABAP system, the next task for me is to develop a function module which accepts github issue source code with markdown format as input, and generate an internal table as output which looks like below:

Use JavaScript regular expression to parse image reference in markdown source code

I have written the following code to parse the image reference in one of my github issue:

const regex = /!\[(.*?)\]\((.*?)\)/g;
const str = `# Created by Jerry Wang, last modified on Jun 24, 2014

User parameter里只要BSPWD_USER_LEVEL 值大于5 即可在webclient UI上显示error message的technical information:

message level > 5的判断:
let m;
var printResult = ( array) => {
    var url = array[2];
    var splited = url.split(".");
console.log("local file: " + array[1] + "." + splited[splited.length-1]);
console.log("url: " + url);
while ((m = regex.exec(str)) !== null) {
    if (m.index === regex.lastIndex) {

The output shows this regular expression works as expected:

Solution in ABAP

When I copy the working regular expression !\[(.*?)\]\((.*?)\) in JavaScript to ABAP, it does not work – this regular expression is complained as invalid.

From this stackoverflow discussion thread Greedy/non-greedy quantifiers in ABAP regular expressions I get to know that the symbol ? for non-greedy match behavior in regular expression is not supported in ABAP.

Thus I have to use some alternative to achieve the same output as in JavaScript.

Approach1 – use dynamically generated regular expression

My idea is to first identify all occurrence of “![xxx](yyyy)” by dynamically generation of corresponding regular expression pattern.

I have written another tool class to achieve this requirement:

  create public .
public section.
    BEGIN OF ty_image_reference,
           image_name TYPE string,
           image_url TYPE string,
         END OF ty_image_reference .
    tt_image_reference TYPE TABLE OF ty_image_reference with key image_name .

  class-methods GET_IMAGE_REFERENCE
      value(RT_IMAGE) type TT_IMAGE_REFERENCE .
protected section.
private section.
  class-data SV_IMAGE_PATTERN type STRING value '(!\[.*\]\(.*\))' ##NO_TEXT.
  METHOD get_image_reference.
    DATA: lv_reg_pattern TYPE string,
          lt_result_tab  TYPE match_result_tab.
    FIND ALL OCCURRENCES OF '![' IN iv_issue_source_code MATCH COUNT DATA(lv_count).
    CHECK lv_count > 0.
    lv_reg_pattern = sv_image_pattern.
    IF lv_count > 1.
      DO lv_count - 1 TIMES.
        lv_reg_pattern = lv_reg_pattern && '.*' && sv_image_pattern.
        FIND ALL OCCURRENCES OF REGEX lv_reg_pattern
             IN iv_issue_source_code
             RESULTS lt_result_tab.
      CATCH cx_root INTO DATA(cx_root).
        WRITE:/ cx_root->get_text( ).
    READ TABLE lt_result_tab ASSIGNING FIELD-SYMBOL(<result>) INDEX 1.
    CHECK sy-subrc = 0.
    LOOP AT <result>-submatches ASSIGNING FIELD-SYMBOL(<match>).
      WRITE:/ 'Match...........'.
      WRITE:/ iv_issue_source_code+<match>-offset(<match>-length).

Test via the following source code and this approach can work as expected:
SELECT SINGLE * INTO @DATA(ls) FROM crmd_git_issue WHERE repo_name = 'KM' AND
   issue_num = 282.
cl_abap_git_issue_image_tool=>get_image_reference( ls-issue_body ).

However if there are too many image references used in a markdown document, the dynamic regular expression generation will end up with exception CX_SY_REGEX_TOO_COMPLEX. As a result I have to seek other alternative solution.

Approach 2 – use ABAP to consume regular expression parse service implemented in JavaScript

As illustrated in JavaScript part that we have already elegant solution to parse local file name and file url via JavaScript, why not directly consume the result done by JavaScript in our ABAP code?

step 1 – upload the JavaScript solution to SCP

As soon as the JavaScript solution is uploaded to SCP ( SAP cloud platform ), it could act as a RESTful Service endpoint which could be consumed by any other programming language.
Please follow this blog of mine Deploy your web application to SAP Cloud Platform which can access resource from On-Premise ABAP system to upload this nodeJs application to SAP cloud platform. Once deployed, access url in browser, and it is expected to see “hello World” response:

step 2 – consume the JavaScript service in ABAP

I have encapsulated the necessary logic of sending http request to SAP cloud platform in utility class cl_abap_git_issue_image_tool ( source code could be found from my github)
Test code:

SELECT SINGLE * INTO @DATA(ls) FROM crmd_git_issue WHERE repo_name = 'KM' AND issue_num = 282.  DATA(lt_image) = cl_abap_git_issue_image_tool=>get_image_ref_via_js_service( ls-issue_body ).

Totally 8 images referenced in this issue are parsed as expected:

No comments:

Post a Comment