polserver/pol-core/bscript/compiler/astbuilder/ValueBuilder.cpp

292 lines
8.8 KiB
C++
Raw Permalink Normal View History

#include "ValueBuilder.h"
#include <cstring>
#include "bscript/compiler/Report.h"
#include "bscript/compiler/ast/BooleanValue.h"
#include "bscript/compiler/ast/FloatValue.h"
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
#include "bscript/compiler/ast/FunctionExpression.h"
#include "bscript/compiler/ast/FunctionReference.h"
#include "bscript/compiler/ast/IntegerValue.h"
Add support for regular expressions (#818) * implementation * tests * maybe fix windows compilation? * undo change of match_flag_type * switch to boost regex * move flags to BRegExp object * fix tests due to cmake 4 update * update grammar * update prettifier * move flag parsing to BRegExp ctor * add AST nodes, update instr generation and execution * copy tests but use regular expression literals * fix grammar for handling division correctly * modify return values a bit, update tests - string.match: make groups hold structs of matched, offset - string.match: add offset - string.replace: use a groups array like string.match vs individual arguments * bundle of changes - standardize error messages - add more tests for coverage - move BRegExp creation to static method: previous implementation had ctor throwing, which was no good inside executor since it didn't have a try/catch * more tests; add OT_REGEXP to basic.em * move string regex stuff to bregexp and support regex/wregex via std::visit * maybe fix compilation errors? * add unicode escape sequence handling * really maybe fix compilation error? * fix compiler warnings * refactor a bit to remove duplicate code * address discord comments - just return wstring, no need for vector<wchar_t> method * Add string.split by string and regexp * Use Max_Split instead of Limit This makes it match basic::SplitWords * add support for empty string delim in SplitWords * remove duplicate code across string.split and mf_SplitWords * Squashed commit of the following: commit b3148e069f4c4bad041b36144d312f375c028a6c Author: turleypol <turley@polserver.com> Date: Sat Sep 27 17:02:22 2025 +0200 memorylocation of input string is not allowed to changed, switched back to uniqueptr commit 5a4d1c8fcb2d8239ba0b693644dd20d9a5a167bb Author: turleypol <turley@polserver.com> Date: Sat Sep 27 10:18:01 2025 +0200 no need to use ptr for input commit 90fa0edc66034db77a43222612b4981bf09081a5 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:56:29 2025 +0200 fixed typo commit bed4ff3ac9de88fb9e91350125339c8ab04c2da3 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:47:36 2025 +0200 make Callback for BContinuation move only * use uninit when group isn't matched * fix multiline flag handling; add test * add docs and doc example tests * some cleanup * address review comments - allow move assignment * add core-changes
2025-11-17 18:02:45 +01:00
#include "bscript/compiler/ast/RegularExpressionValue.h"
#include "bscript/compiler/ast/StringValue.h"
#include "bscript/compiler/ast/UninitializedValue.h"
#include "bscript/compiler/astbuilder/BuilderWorkspace.h"
#include "bscript/compiler/file/SourceLocation.h"
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
#include "bscript/compiler/model/CompilerWorkspace.h"
#include "bscript/compiler/model/FunctionLink.h"
#include "clib/strutil.h"
2020-08-19 10:43:08 -07:00
using EscriptGrammar::EscriptParser;
namespace Pol::Bscript::Compiler
{
ValueBuilder::ValueBuilder( const SourceFileIdentifier& source_file_identifier,
BuilderWorkspace& workspace )
: TreeBuilder( source_file_identifier, workspace ), current_scope_name( ScopeName::Global )
{
in_constructor_function.push( false );
}
std::unique_ptr<BooleanValue> ValueBuilder::bool_value(
EscriptGrammar::EscriptParser::BoolLiteralContext* ctx )
{
bool value;
auto loc = location_for( *ctx );
if ( ctx->BOOL_TRUE() )
{
value = true;
}
else if ( ctx->BOOL_FALSE() )
{
value = false;
}
else
{
location_for( *ctx ).internal_error( "unhandled boolean literal" );
}
return std::make_unique<BooleanValue>( loc, value );
}
2020-08-19 10:43:08 -07:00
std::unique_ptr<FloatValue> ValueBuilder::float_value(
EscriptGrammar::EscriptParser::FloatLiteralContext* ctx )
{
auto loc = location_for( *ctx );
auto terminal = ctx->FLOAT_LITERAL();
if ( !terminal )
{
terminal = ctx->HEX_FLOAT_LITERAL();
if ( !terminal )
{
location_for( *ctx ).internal_error( "unhandled float literal" );
}
}
double value = std::stod( text( terminal ) );
2020-08-19 10:43:08 -07:00
return std::make_unique<FloatValue>( loc, value );
}
std::unique_ptr<FunctionReference> ValueBuilder::function_reference(
EscriptParser::FunctionReferenceContext* ctx )
{
auto source_location = location_for( *ctx );
auto name = text( ctx->function );
auto scope = ctx->scope ? ScopeName( text( ctx->scope ) ) // scope exists
: ctx->COLONCOLON() ? ScopeName::Global // colon exists
: ScopeName::None; // has no scope specified
auto function_link =
std::make_shared<FunctionLink>( source_location, current_scope_name.string() );
auto function_reference =
std::make_unique<FunctionReference>( source_location, name, function_link );
workspace.function_resolver.register_function_link( ScopableName( scope, name ), function_link );
return function_reference;
}
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
std::unique_ptr<FunctionExpression> ValueBuilder::function_expression(
EscriptGrammar::EscriptParser::FunctionExpressionContext* ctx )
{
auto loc = location_for( *ctx );
auto name = workspace.function_resolver.register_function_expression( loc, ctx );
workspace.compiler_workspace.all_function_locations.emplace( name, loc );
workspace.function_resolver.force_reference( ScopableName( ScopeName::Global, name ), loc );
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
auto function_link =
std::make_shared<FunctionLink>( loc, current_scope_name.string() /* calling scope */ );
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
// A unique name, as it is based on source location, so registering the link
// in global scope is okay.
workspace.function_resolver.register_function_link( ScopableName( ScopeName::Global, name ),
function_link );
Add compiler support for function expressions (#671) * initial poc of function expressions - update grammar - mock AST builder to return BBoolean(true) for a func expr - update prettifier for skeleton implementation * more skeleton work create AST class FunctionExpression mimicking boolean value * create ast UserFunction, add to workspace from functexpr * can generate instructions * reorg tests; add test for instructions * can track FunctionDepth in Variable * can get captures for funcexprs inside funcs.. tbd if this way of nesting works * Implement create-functor instruction - Move function depth from Variable to Variables - Introduce stacking of `Variables` for function expresions via `FunctionVariableScope` - Add `TOK_FUNCTOR` instruction for 'create-functor' - Handle emitting a `FunctionExpression` AST node - Update `emit.declare_variable` and `emit.access_variable` to account for function captures - Remove `in_function` from instruction generator, as it is tracked via `UserFunction` stack - Update tests * Address Discord comments - Swap pop param order * Bubble up captured variables through nested functions * Improve testing infrastructure; add some test cases * Fix compilation error * prepend captures to function parameters in funcref mth_call * update tests * move from function{} to @{} * implementation fix * some more tests * update docs * Some cleanup * fix CI annotation warning - 'argument': conversion from 'size_t' to 'VariableIndex', possible loss of data * Address review comments - Add `passert_always` - Use better example in docs * Some cleanup - Remove unused functions
2024-07-29 22:30:52 +02:00
return std::make_unique<FunctionExpression>( loc, function_link );
}
2020-08-25 23:27:24 -07:00
std::unique_ptr<IntegerValue> ValueBuilder::integer_value(
EscriptParser::IntegerLiteralContext* ctx )
{
auto loc = location_for( *ctx );
return std::make_unique<IntegerValue>( loc, to_int( ctx ) );
}
std::unique_ptr<StringValue> ValueBuilder::string_value( antlr4::tree::TerminalNode* string_literal,
bool expect_quotes )
{
auto loc = location_for( *string_literal );
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
return std::make_unique<StringValue>( loc, unquote( string_literal, expect_quotes ) );
}
Add support for regular expressions (#818) * implementation * tests * maybe fix windows compilation? * undo change of match_flag_type * switch to boost regex * move flags to BRegExp object * fix tests due to cmake 4 update * update grammar * update prettifier * move flag parsing to BRegExp ctor * add AST nodes, update instr generation and execution * copy tests but use regular expression literals * fix grammar for handling division correctly * modify return values a bit, update tests - string.match: make groups hold structs of matched, offset - string.match: add offset - string.replace: use a groups array like string.match vs individual arguments * bundle of changes - standardize error messages - add more tests for coverage - move BRegExp creation to static method: previous implementation had ctor throwing, which was no good inside executor since it didn't have a try/catch * more tests; add OT_REGEXP to basic.em * move string regex stuff to bregexp and support regex/wregex via std::visit * maybe fix compilation errors? * add unicode escape sequence handling * really maybe fix compilation error? * fix compiler warnings * refactor a bit to remove duplicate code * address discord comments - just return wstring, no need for vector<wchar_t> method * Add string.split by string and regexp * Use Max_Split instead of Limit This makes it match basic::SplitWords * add support for empty string delim in SplitWords * remove duplicate code across string.split and mf_SplitWords * Squashed commit of the following: commit b3148e069f4c4bad041b36144d312f375c028a6c Author: turleypol <turley@polserver.com> Date: Sat Sep 27 17:02:22 2025 +0200 memorylocation of input string is not allowed to changed, switched back to uniqueptr commit 5a4d1c8fcb2d8239ba0b693644dd20d9a5a167bb Author: turleypol <turley@polserver.com> Date: Sat Sep 27 10:18:01 2025 +0200 no need to use ptr for input commit 90fa0edc66034db77a43222612b4981bf09081a5 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:56:29 2025 +0200 fixed typo commit bed4ff3ac9de88fb9e91350125339c8ab04c2da3 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:47:36 2025 +0200 make Callback for BContinuation move only * use uninit when group isn't matched * fix multiline flag handling; add test * add docs and doc example tests * some cleanup * address review comments - allow move assignment * add core-changes
2025-11-17 18:02:45 +01:00
std::unique_ptr<RegularExpressionValue> ValueBuilder::regular_expression_value(
antlr4::tree::TerminalNode* regular_expression_literal )
{
std::string input = regular_expression_literal->getSymbol()->getText();
std::string pattern;
std::string flags;
std::size_t pos = input.rfind( '/' );
if ( input.size() < 2 || input[0] != '/' || pos == std::string::npos )
{
location_for( *regular_expression_literal )
.internal_error( "regular expression does not begin with a slash?" );
}
pattern = input.substr( 1, pos - 1 );
flags = input.substr( pos + 1 );
auto loc = location_for( *regular_expression_literal );
return std::make_unique<RegularExpressionValue>( loc, std::move( pattern ), std::move( flags ) );
}
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
std::string ValueBuilder::unquote( antlr4::tree::TerminalNode* string_literal, bool expect_quotes )
{
std::string input = string_literal->getSymbol()->getText();
const char* s = input.c_str();
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
if ( *s != '\"' && expect_quotes )
location_for( *string_literal ).internal_error( "string does not begin with a quote?" );
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
const char* end = s + ( expect_quotes ? 1 : 0 );
std::string lit;
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
lit.reserve( input.length() );
bool escnext = false; // true when waiting for 2nd char in an escape sequence
u8 hexnext = 0; // tells how many more chars in a \xNN escape sequence
char hexstr[3]; // will contain the \x escape chars to be processed
memset( hexstr, 0, 3 );
for ( ;; )
{
if ( !*end )
{
Add support for interpolated strings ("interstrings") (#369) * Update grammar to support interpolated strings * Struct initializers, case switchs are regular strings * Initial stub for interpolated string in AST This splits the previous STRING_LITERAL terminal into a production, stringLiteral, to handle REGULAR_STRING terminal and interpolatedString production. * Change interpolated strings to expressions * Fix mode handling in grammar * Grammar fixes - Fix mode handling in lexer - Interpolated strings only have one expression * Can successfully parse, stub compile interstrings $"He {there + print("hello")}"; * Add formatting string to grammar * Fix tests, and add test stubs * Rename curleys to brace; better AST generation * initial work interstring instruction * Initial work on formatted string instruction: emit * Formatted string instruction: execution * Add simple formatted string test * Add formatted string test src * Review changes #1 - Rename `FormattedString` to `FormatExpression` - Rename overloaded `try_to_format()` to `get_formatted()` * Review changes part 2 - Rename `InterpolatedString` ast node to `InterpolateString` * Review changes part 3 - Modify `ins_interpolate_string` * Review changes part 4 - Remove unused formal parameter * Add negative tests * Spruce up positive tests * Fix braces, escape chars inside interstring * Fix, add tests for escaped char and double brace * Address differences for new lines in tests * Update Escript version * Make empty expression an error * Add documentation for interpolated strings * Quick touchup on docs
2021-03-11 22:48:16 +01:00
if ( !expect_quotes )
break;
// parser should catch this.
location_for( *string_literal ).internal_error( "unterminated string" );
}
if ( escnext && hexnext )
location_for( *string_literal )
.internal_error( "Bug in the compiler. Please report this on the forums." );
if ( escnext )
{
// waiting for 2nd character after a backslash
escnext = false;
if ( *end == 'n' )
lit += '\n';
else if ( *end == 't' )
lit += '\t';
else if ( *end == 'x' )
hexnext = 2;
else
lit += *end;
}
else if ( hexnext )
{
// waiting for next (two) chars in hex escape sequence (eg. \xFF)
hexstr[2 - hexnext] = *end;
if ( !--hexnext )
{
char* endptr;
char ord = static_cast<char>( strtol( hexstr, &endptr, 16 ) );
if ( *endptr != '\0' )
{
report.error( location_for( *string_literal ), "Invalid hex escape sequence '{}'.",
hexstr );
return lit;
}
lit += ord;
}
}
else
{
if ( *end == '\\' )
escnext = true;
else if ( *end == '\"' )
break;
else
lit += *end;
}
++end;
}
if ( !Clib::isValidUnicode( lit ) )
{
report.warning( location_for( *string_literal ),
"Warning: invalid unicode character detected. Assuming ISO8859." );
Clib::sanitizeUnicodeWithIso( &lit );
}
return lit;
}
2020-08-19 10:43:08 -07:00
std::unique_ptr<Value> ValueBuilder::value( EscriptParser::LiteralContext* ctx )
{
if ( auto string_literal = ctx->STRING_LITERAL() )
{
return string_value( string_literal );
}
if ( auto integer_literal = ctx->integerLiteral() )
2020-08-25 23:27:24 -07:00
{
return integer_value( integer_literal );
}
if ( auto float_literal = ctx->floatLiteral() )
2020-08-19 10:43:08 -07:00
{
return float_value( float_literal );
}
if ( auto bool_literal = ctx->boolLiteral() )
{
return bool_value( bool_literal );
}
if ( ctx->UNINIT() )
{
return std::make_unique<UninitializedValue>( location_for( *ctx ) );
}
if ( auto regex = ctx->REGEXP_LITERAL() )
Add support for regular expressions (#818) * implementation * tests * maybe fix windows compilation? * undo change of match_flag_type * switch to boost regex * move flags to BRegExp object * fix tests due to cmake 4 update * update grammar * update prettifier * move flag parsing to BRegExp ctor * add AST nodes, update instr generation and execution * copy tests but use regular expression literals * fix grammar for handling division correctly * modify return values a bit, update tests - string.match: make groups hold structs of matched, offset - string.match: add offset - string.replace: use a groups array like string.match vs individual arguments * bundle of changes - standardize error messages - add more tests for coverage - move BRegExp creation to static method: previous implementation had ctor throwing, which was no good inside executor since it didn't have a try/catch * more tests; add OT_REGEXP to basic.em * move string regex stuff to bregexp and support regex/wregex via std::visit * maybe fix compilation errors? * add unicode escape sequence handling * really maybe fix compilation error? * fix compiler warnings * refactor a bit to remove duplicate code * address discord comments - just return wstring, no need for vector<wchar_t> method * Add string.split by string and regexp * Use Max_Split instead of Limit This makes it match basic::SplitWords * add support for empty string delim in SplitWords * remove duplicate code across string.split and mf_SplitWords * Squashed commit of the following: commit b3148e069f4c4bad041b36144d312f375c028a6c Author: turleypol <turley@polserver.com> Date: Sat Sep 27 17:02:22 2025 +0200 memorylocation of input string is not allowed to changed, switched back to uniqueptr commit 5a4d1c8fcb2d8239ba0b693644dd20d9a5a167bb Author: turleypol <turley@polserver.com> Date: Sat Sep 27 10:18:01 2025 +0200 no need to use ptr for input commit 90fa0edc66034db77a43222612b4981bf09081a5 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:56:29 2025 +0200 fixed typo commit bed4ff3ac9de88fb9e91350125339c8ab04c2da3 Author: turleypol <turley@polserver.com> Date: Sat Sep 27 09:47:36 2025 +0200 make Callback for BContinuation move only * use uninit when group isn't matched * fix multiline flag handling; add test * add docs and doc example tests * some cleanup * address review comments - allow move assignment * add core-changes
2025-11-17 18:02:45 +01:00
{
return regular_expression_value( regex );
}
location_for( *ctx ).internal_error( "unhandled literal" );
2020-08-19 10:43:08 -07:00
}
2020-08-25 23:27:24 -07:00
int ValueBuilder::to_int( EscriptParser::IntegerLiteralContext* ctx )
{
try
2020-08-25 23:27:24 -07:00
{
if ( auto decimal_literal = ctx->DECIMAL_LITERAL() )
{
return std::stoi( decimal_literal->getSymbol()->getText() );
}
if ( auto hex_literal = ctx->HEX_LITERAL() )
{
return static_cast<int>( std::stoul( hex_literal->getSymbol()->getText(), nullptr, 16 ) );
}
if ( auto oct_literal = ctx->OCT_LITERAL() )
{
return std::stoi( oct_literal->getSymbol()->getText(), nullptr, 8 );
}
if ( auto binary_literal = ctx->BINARY_LITERAL() )
{
return std::stoi( binary_literal->getSymbol()->getText(), nullptr, 2 );
}
2020-08-25 23:27:24 -07:00
}
catch ( std::invalid_argument& )
2020-08-25 23:27:24 -07:00
{
report.error( location_for( *ctx ), "unable to convert integer value '{}'.", ctx->getText() );
throw;
2020-08-25 23:27:24 -07:00
}
catch ( std::out_of_range& )
2020-08-25 23:27:24 -07:00
{
report.error( location_for( *ctx ), "integer value '{}' out of range.", ctx->getText() );
throw;
2020-08-25 23:27:24 -07:00
}
2020-08-25 23:27:24 -07:00
location_for( *ctx ).internal_error( "unhandled integer literal" );
}
} // namespace Pol::Bscript::Compiler